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DOCUMENT-IDENTIFIER: US 5990990 A 

TITLE: Three-dimensional display techniques, device, systems and method of 
presenting data in a volumetric format 



Brief Summary Text (17): 

It is another object of the invention to provide light manipulating devices, namely 
display forming devices, volumetric three-dimensional image display forming and 
projection devices, laser light apparatus, light processors and projectors, systems 
and components thereof, for use to generate, manipulate and display three- 
dimensional spatial images of objects or data in the following markets and for the 
following applications: science, entertainment, virtual reality, computer-aided 
design, computer-aided manufacturing, computer-aided engineering, aerospace 
simulation, air traffic control, and optical computing. 

Detailed Description Text (36) : 

Evidently, in viewing a "virtual" image (e.g., 730a) from one viewpoint, the viewer 
(760) only makes use of a small portion of the reflector surfaces. This presents 
the possibility that only a reduced portion or "sector" of an image translation 
chamber is required to view a three-dim en sional virtual image from a single 
viewpoint. An image translation chamber "segment" suited to single or restricted 
viewpoint viewing of an image is described with respect to FIG. 7B. 

Detailed Description Text (51) : 

The discussion hereinabove with respect to FIGS. 7A, 73 and 8A-E has been directed 
to techniques of image translation whereby a "real" image is perceived by a viewer 
as a "virtual" image at a different location. These techniques manipulate three- 
dimensional images which have already been created. Attention is now directed to 
techniques aimed at generating images, including color images. 

Detailed Description Text (145) : 

FIGS. 26A, 26B and 26C illustrate embodiments of volumetric display devices wherein 
the HOE is disposed generally vertically, rather than horizontally. In these 
embodiments, the HOE can be used to "wallpaper" a viewing room, enabling a 
plurality of viewers to v_i_ew a three-dimensional image. It is generally preferred 
in these embodiments to use multiple (at least two) projectors which project 
multiple iterations of the input image at the HOE wallpaper from multiple angles. 
Evidently, the multiple images need to be correlated with one another since they 
are, in essence, views taken from different angles. The following figures describe 
using HOEs and transmission display devices employing multiple projection sources 
and yielding a wide (up to 360. degree.) viewing range suitable for multiple-user 
applications . 

Detailed Description Text (182) : 

FIG. 32A shows an AO (acousto-optic) line scanner 3200. The scanner is designed to 
perform the task of scanning, at an extremely rapid rate. Generally, an AO driver 
(3210) sends shear waves down a TeO.sub.2 (or other suitable AO) crystal, making a 
pattern of points, or COLUMNS (all the waves are staggered at different spacings) . 
A string (sequence) of COLUMNS in this pattern becomes a single row of on/off 
points (analogous to one vertical row on a television) . This ROW is bounced off of 
a line motor scanner (3212, 3214), designed to perform this function at a very fast 
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rate without requiring random access, then advances to the next ROW below (behind, 
in the sequence) it. This enables raster-type scanning, as indicated by the pattern 
3218 on the screen 3220. In a sense, a laser (3202) is used like an automotive 
timing light (e.g., to freeze motion), in this case to isolate each of the ROWs of 
information (shear waves) that are travelling down the crystal (3210) at the speed 
of sound. The laser beam may be doubled in a YAG doubling chamber 3204 prior to 
being made wedge-shaped by a cylinder optic (lens) 3205. The wedge-shaped beam is 
collimated by a collimator optic (lens) 3206) prior to being acted upon by the AO 
crystal 3210. The AO crystal 3210 is driven by a radio frequency (RF) signal (not 
shown) . A vertical axis galvo scanner 3212 drives (rotates) a reflective paddle 
3214, to provide discrete "light bytes" (tm), on a row-by-row basis, to a screen 
3220, where the volumetric data can be raster (alternatively, vector) scanned 
(3218) and emitted along a path 3222 to emitter optics and an MOE (not shown) . The 
screen 3220 functions analogously to a television screen, and the system 3200 is 
useful for x,y scanning of enormous volumes of data at a very rapid rate, for 
subsequent formatting (by emitter optics, MOE, and the like) into a volumetric 
format. Such high-density, volumetrically-f ormatted data, can be transmitted over a 
wide link, such as a fiber optic cable (not shown) , to be decoded (disassembled, 
de-volumetrized) at the receiving end. 

Current US Class (1) ; 
345 
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Computer-Aided Structural Engineering (CASE) Project: Computer- 
Aided Structural Modeling (CASM). Version 6.00, Report 2. User's 
Guide 
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Abstract: The Computer-Aided Structural 
Modeling (CASM) computer program is designed 
to aid the structural engineer in the preliminary 
design and evaluation of structural building 
systems by the use of three dimensional (3-D) 
interactive graphics. CASM allows the structural 
engineer to quickly evaluate various framing 
alternatives in order to make more Informed 
decisions in the initial structural evaluation 
process. The program was developed by the 
Information Technology Laboratory in conjunction 
with the Computer-Aided Structural Engineering 
(CASE) Project, Building Systems Task Group. 
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Three Dimensional Design Tutorial 

At present, I introduce three dimensional modeling with Infini-D in my sculpture ... This 
could include areas such as design, architecture, sculpture, ... 
wvw.colleges.org/techcenter/Fellowships/grants/1998/visser.html - 8k - 
Cached - Similar pages 

Three-Dimensional Computer Graphics Architecture - Mitra, Chiueh .„ 
Three dimensional D computer graphics hardware has emerged to become an integral ... 
2: The Design and Analysis of a Cache Architecture for Texture Mapping ... 
citeseer.ist.psu.edu/mitraOOthreedimensional.html - 25k - Cached - Similar pages 
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SketchUp 

From Wikipedia, the free encyclopedia 

SketchUp is a 3D modeling program designed for 
professional architects, civil engineers, filmmakers, 
game developers, and related professions. It is also 
widely used by teens and pre-teens as an 
entertainment software. Most people however use it to 
design buildings to be displayed on Google Earth. It 
was designed to be more intuitive, flexible, and easy- 
to-use than other 3D modeling programs, which often 
have steep learning curves. [1] 
(http://www,sketchup.com/?id=3) Several features 
allow designers to "play" with their designs, unlike 
what is possible in other 3D CAD programs. It is 
marketed as an easy-to-use conceptual tool with a 
simple interface. A well known feature in the 
SketchUp World is the 3D Warehouse. Users can use 
their Google Accounts to upload models, and then 
browse the 3D Warehouse for many components and models. 

Some of its key features and uses include: 

■ A "smart" drawing cursor (inference) system that allows users to draw 3D objects using a 2D 
screen and mouse. 

■ Simple massing study capability via "push-pull". 

■ An interactive Heliodon, or sun angle simulator. 

■ Able to animate camera and sun movements. 

■ Models can be individually colored with an assortment of solid colors, textures and materials. 

■ Can be used with Google Earth. 

■ Useftil and generally easier to use than other modeling programs for children, amateur artists, car 
enthusiasts and other hobbyists. 
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History 

SketchUp was developed by startup company @Last 
Software, Boulder, Colorado which was formed in 

1999. [2] (http://www.sketchup.com/?id=3) SketchUp 
was first released in August 2000 [3] 
(http://www.sketchup.com/?id=4&recordid=3) as a 
general purpose 3D content creation tool. It won a 
Community Choice Award at its first tradeshow in 

2000. (A/E/C SYSTEMS Fall 2000 




A screenshot of Google Sketchup. It 
demonstrates its powerful modeling abilities and 
ease-of-use. 



(http://www.digitalcad.com/Htm/Articles/2000/ll_00b/digicad.htm)) 
[4] (http://www.sketchup.com/?id=4&recordid=l) It quickly found a 
market in architecture and building design industries, however, and 
was revised to suit the needs of working professionals. The key to its 
early success was in its fast learning curve, allowing for a shorter 
leaming period than other commercially available 3D tools. 

As of SketchUp 5, features were added to allow the user to extrude and 
widen as well as the ability for a face to "follow" the cursor around an 
object. 




A screenshot of SketchUp 3 in 
Mac 



On March 14, 2006, Google acquired @Last Software,[5] (http://www.sketchup.com/index.php? 
id=7&recordid=109) attracted by their plugin for Google Earth. 
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On January 9, 2007, SketchUp 6 was released, featuring a handful of new tools as well as a beta version 
of Google SketchUp LayOut. Lay Out includes a handful of 2D vector tools, as well as page layout tools 
making it easier for professionals to create presentations without jumping to a third-party presentation 
program. 

On February 9, 2007, a maintenance update was released. It corrects a number of bugs, but brings no 
new features. 



Add-ons can be downloaded from the SketchUp Web Site (http://wv^.sketchup.com/)'s Plugins page 
(http://vmw.sketchup.com/?sid=37). 

Google Earth 

One such add-on allows the user to export the 3D model as a .kmz file which can then be opened in 
Google Earth. This allows accurate geo-referencing of SketchUp models and accurate placement of 
those models in Google Earth. It also allows SketchUp to import modeling context (photographic + 
terrain model) from Google Earth for modeling reference. 

Other Add-ons 

Other add-ons are provided for export to: 

■ 3D Text (http://www.suwiki.org/suwiki/index.php?title=3D_Text_Tutorial) — for creating 3D 
text. 

■ 3dsMax 

■ Construction Documentation (http://forums.renderplus.com/wiki/index.php? 
title=Construction_Documentation) for create 2D plan and elevation documents. 

■ IRender (http://forums.renderplus.com/wiki/index.php?title=IRender) — fully integrated Ray 
Trace Renderer for SketchUp using the AccuRender engine. 

■ RPS 3D PDF (http://www.renderplus.com/pdf/pdfj)roduct.htm) for interactive 3D PDF 
documents. 



Add-Ons 



Tools 



Drawing tools 



This tool can create an arc from two points 
and a drag. 



Line 



This is used to draw a regular straight line. 



Arc 
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Freehand 

This allows the user to scribble and create a 
freehand figure. 

Circle 

The user can make circles by clicking a 
point and dragging out the radius. 

Polygon 

Polygons are constructed in much the same 
way as circles; they are clicked and dragged. 

Rectangle 

This tool is used to make rectangles with by 
clicking and dragging to adjust the length 
and width. 

Editing tools 
Push/pull 

The push/pull is a tool that grabs a two-dimensional surface and pushes or pulls it into three dimensions. 
This, google claims in the tutorial, is the main idea of google sketchUp. U.S. patent #6,628,279 is a 
patent for this technology. 

Rotate 

A protractor is used to angle selected geometrical pieces. 
Move 

The user's mouse can select and move pieces on the three different axes or on some combination of the 
three. 

Link 

Other useful tools can be found here (http://www.suwiki.org/suwiki/index.php? 
title=Image:MainTools3 .png) 

Patents 

SketchUp holds a Patent U.S. Patent 6,628,279 (http://patft.uspto.gov/netacgi/nph-Parser? 
patentnumber=6628279) on its Push/Pull (http://www.suwiki.org/suwiki/index.php?title=Push/Pull) 




User-friendly lools facilitate the creation of 3D models. 



Many different 3D and 2D exporters are available in 
SketchUp for uses such as rendering. 
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technology: 

"System and method for three-dimensional modeling: A three-dimensional design and modeling 
environment allows users to draw the outlines, or perimeters, of objects in a two-dimensional manner, 
similar to pencil and paper, already familiar to them. The two-dimensional, planar faces created by a 
user can then be pushed and pulled by editing tools within the environment to easily and intuitively 
model three-dimensional volumes and geometries." 

The patent was applied for in November 2000, and awarded in September 2003 (U.S. Patent 6,628,279 
(http://patft.uspto.gov/netacgi/nph-Parser?patentnumber=6628279)). 

Google SketchUp 

On April 27, 2006, Google announced Google SketchUp, a freely- 
downloadable version of SketchUp. [6] (http://sketchup.googlexom/) The 
free version is not as capable as SketchUp Pro 5, but it includes integrated 
tools for uploading content to Google Earth and to the Google 3D 
Warehouse,[7] (http://sketchup.google.com/3dwarehouse/) a library of 
models created in SketchUp. 

It can export 3D Google Earth .kmz files and 2D graphic: .bmp .png .jpg .tif. 
Google SketchUp Pro can export more 3D graphic files: 
.3ds .dwg .dxf .fbx .obj .xsi .wy\ and more 2D graphic: 
.pdf .eps .epx .dwg .dxf. 

Ruby Application Programming Interface (API) 

Ruby is a scripting language that was first introduced in SketchUp 4. Since its introduction SketchUp 
users have shovm their creativity by creating hundreds of useful tools using SketchUp Ruby. 

(For Google SketchUp) You can use the Hello World Tutorial 

(http://www.suwiki.org/suwiki/index.php?title=Ruby_Tutorial_-_Hello_World) to create dialog boxes. 

User community 

Gallery 

SketchUp users can share their work for free in the Gallery (http://www.sketchup.com/?sid=485) on the 
SketchUp official website (http://vAvw.sketchup.com/). This gives a place where users can share their 
work in many fields including, Architecture, Construction, Education K-12, Game Development, GIS, 
Graphic Art, Higher education, Industrial Design / Furniture, Interior Design, Landscape Architecture, 
and Mechanical Design. 

SketchUcation 

SketchUcation (http://www.sketchucation.com/) is a gateway to the SketchUp Community. Information 
on worldwide training locations, Training CDs & DVDs and online training. But also tutorials on the 




A 3d model made using 
the free version of 
Google SketchUp 
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User Interface of SketchUp such as the Tools and Menus as well as a growing number of function- 
specific tutorial videos or how to model objects. There is also a Blog and a Forum, Coming in the Fall of 
2007 is a Game Development subsite. Other sections such as SketchUp add-ons and tutorials on the 
Panels and VCB will be added soon. 

SuWiki 

SuWiki (http://suwiki.org/) is a Wiki encyclopedia for SketchUp users. It provides an open environment 
for SketchUp users to find, and contribute to article and tutorials on the use of SketchUp. 

Forums 

Many users also share their ideas and knowledge about SketchUp and other modeling programs in the 
SketchUp Pro Forum (http://forum.sketchup.com/forumdisplay .php?f=l). 

There is a Google Group Forum (http://groups.google.com/group/SketchUp/topics) for users of Google 
SketchUp (free). 

The SketchUp Pro Community Forum (http://www.sketchup-forum,com/) is a portal dedicated to 
SketchUp Pro users only and does focus solely on user communication by providing a modem, 
configurable interface combined with a powerful forum, gallery and download repository. 

Plugins 

SketchUp maintains a web site of plugins (http://www.sketchup.com/?sid=37) for SketchUp. Both 3rd 
party plugins and plugins developed by SketchUp. 

External links 

■ SketchUp Homepage (http://www.sketchup.com/) 

■ 3D Warehouse for exchanging models via Google Earth 
(http://sketchup.google.com/3dwarehouse/) 

■ SketchUp Video Tutorials (http://sketchup.google.com/vtutorials.html) 

■ SketchUp Google Group (http://groups.google.com/group/SketchUp) 

■ The SketchUp Components Collection (http://scc.jazzle.co.uk/) 

■ www.sketchucation.com (http://www.sketchucation.com/) SketchUcation SketchUp Tutorials 

■ www.SUWiki.org (http://www.suwiki.org/) The SketchUp Wiki 

■ SketchUp for Garden Designers and Landscapers (http://www.gardendesignpro.co.uk/) 

■ Exporter for room acoustic modelling (http://www.rahe-kraft.de/cms/su2catt/) 

Retrieved from "http://en.wikipedia.org/wiki/SketchUp" 
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^ Session 3: User-level transactional pro g ranriming in Haskell 
^ Peter Thiemann 

^ September 2006 Proceedings of the 2006 ACM SIGPLAN workshop on Haskell Haskell 
■06 

Publisher: ACM Press 

Full text available: ^pdf(212.38 KB) Additional Information: full citation , abstract , references , index terms 

Correct handling of concurrently accessed external resources is a demanding problem in 
programming. The standard approaches rely on database transactions or concurrency 
mechanisms like locks. The paper considers two such resources, global variables and 
databases, and defines transactional APIs for them in Haskell. The APIs provide a novel 
flavor of user-level transactions which are particularly suitable In the context of web- 
based systems. This suitability is demonstrated by providing a s ... 

2 Critical tokenization and its properties 
Jin Guo 

December 1997 Computational Linguistics, volume 23 issue 4 

Publisher: MIT Press 

Full text available: ~ rS 

^.Qai( /.U4 M» }„^ Additional Information: full citation , abstract , references , citin gs 
Publisher Site 

Tokenization Is the process of mapping sentences from character strings into strings of 
words. This paper sets out to study critical tokenization, a distinctive type of tokenization 
following the principle of maximum tokenization. The objective in this paper Is to develop 
its mathematical description and understanding. The main results are as follows: (1) 
Critical points are all and only unambiguous token boundaries for any character string on 
a complete dictionary; (2)Any critically tokenized wo ... 

3 OOPSLA onward! track chair's welcome: JDA: a step towards large-scale reuse on 
^ the web 

^ Seung Chan Slim Lim, Peter Lucas 

October 2006 Companion to the 21st ACM SIGPLAN conference on Object-oriented 
programming systems, languages, and applications OOPSLA '06 

Publisher: ACM Press 

Full text available: ^.edf(§&Z.52_KB) Additional Information: full citation, abstract. Ceferences, index.terms 

The Web Is maturing as a rich application development platform, and efforts are being 
made to provide richer and more dynamic interactions using JavaScript. JavaScript-based 
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Web applications such as Google Maps have gained extra attention because they can be 
easily included in HTML for reuse. Unfortunately, various technical hurdles have made it 
difficult for JavaScript reuse to extend beyond its current state. Furthermore, JavaScript 
reuse is still out of reach for a large portion of the Web us ... 

Keywords: JavaScript, black box reuse, composition, dataflow architecture, information 
devices architecture, rich web application, web engineering 



4 A connputerized analysis of honne ener g y reduction alternatives used in a statewide Q 
extension program 

Robert L. Fehr, George M. Turner, George A. Duncan 

January 1977 Proceedings of the 9th conference on Winter simulation - Volume 2 
WSC '77 

Publisher: Winter Simulation Conference 

Full text available: pdf(668.66 KB) Additional Information: full citation , abstract , references , index terms 

A computer program has been developed which can provide homeowners with an 
evaluation of the economics of adding insulation to the celling, walls and floors of their 
homes along with the addition of storm windows and doors. This evaluation is done from 
data collected on a simplified input form. The basic calculation procedure used in the 
program is discussed in detail including the assumptions made. The program input form 
and typical output are also presented. 



5 Constructin g colJaboiatiye_desktop storage caches fpj large sc^ntific datasets 




Sudharshan S. Vazhkudai, Xiaosong Ma, Vincent W. Freeh, Jonathan W. Strickland, Nandan 
Tammineedi, Tyler Simon, Stephen L. Scott 

August 2006 ACM Transactions on Storage (TOS), volume 2 issue 3 
Publisher: ACM Press 

Full text available: pdf(833.76 KB) Additional Information: full citation , abstract , references , index terms 

High-end computing is suffering a data deluge from experiments, simulations, and 
apparatus that creates overwhelming application dataset sizes. This has led to the 
proliferation of high-end mass storage systems, storage area clusters, and data centers. 
These storage facilities offer a large range of choices in terms of capacity and access rate, 
as well as strong data availability and consistency support. However, for most end-users, 
the "last mile" in their analysis pipeline o ... 

Keywords: Distributed storage, parallel I/O, scientific data management, serverless 
storage system, storage cache, storage networking, storage resoucce management, 
storage scavenging, striped storage 



6 Real world applications: Incorporating fuzzy knowledge into fitness: multiobjective 




evolutionan/ 3D design of process plants 
Ingo MIerswa 

June 2005 Proceedings of the 2005 conference on Genetic and evolutionary 

computation GECCO '05 
Publisher: ACM Press 

Full text available: Q p df (336, 1 7 KB) Additional Information: full citation , abstract , references , index terms 

Designing teciinical plants is a complex and demanding process. It lias been shown that 
the optimization of the simple facility placement problem is already NP-hard. Optimization 
of plant designs must obey a number of criteria derived from several fields of process 
engineering. We discuss an expansion of the simple facility placement problem with non- 
regular floor spaces and more than one layer. Additionally, we allow forbidden zones and 
predefined ways. In contrast to other approaches our system ... 
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Exploitin g components: Exception handlin g in component composition with the | 
^ sup port of mi ddlewa re 

^ Yaodong Feng, Gang Huang, Yali Zhu, Hong Mel 

September 2005 Proceedings of the 5th international workshop on Software 

engineering and middleware SEM '05 
Publisher: ACM Press 

Full text available- IB Ddf (368 18 KB) A^^*^'^"^' Information: full citation , abstract , references , citings, index 

: terms 

Exception handling is a practical and Important way to improve the reliability of a software 
system. In component based software engineering, exceptions are usually handled by the 
developers of reusable components. However, many exceptions are dependent on the 
context of a component in the target system. Such exceptions are difficult and sometimes 
impossible to be handled at the stage of development of reusable components. On the 
other hand, we found out that many exceptions can be successfully ... 

Keywords: component composition, exception handling, middleware 



8 Middleware su p port for auditing service process flows Q 
Hakan Hacigumu§ 

November 2006 Proceedings of the 1st workshop on Middleware for Service Oriented 
Computing (MW4SOC 2006) MW4SOC '06 

Publisher: ACM Press 

Full text available: ^ pdf(141.53 KB) Additional Information: fuli citation , abstract , references , index terms 

The ever increasing security and privacy concerns require organizations to strictly comply 
with the regulations and the guidelines. Similar to any other part of an organization, it is 
necessary to conduct audits to determine whether the IT systems are in compliance. We 
specifically focus on the IT systems that are founded on Service Oriented Computing 
(SOC) principals. In a service oriented system both the internal and the external services 
may easily interact with each other, which may make the ... 

Keywords: auditing, middleware, privacy, security, service oriented computing 



9 New Frontiers of Reverse Engineering Q 
Gerardo CanforaHarman, Massimiliano Di Penta 

May 2007 2007 Future of Software Engineering FOSE *07 
Publisher: IEEE Computer Society 

Full text available: ^ pdf( 396.68 KB ) Additional Information: full citation , abstract 

Comprehending and modifying software is at the heart of many software engineering 
tasks, and this explains the growing interest that software reverse engineering has gained 
in the last 20 years. Broadly speal<ing, reverse engineering is the process of analyzing a 
subject system to create representations of the system at a higher level of abstraction. 
This paper briefly presents an overview of the field of reverse engineering, reviews main 
achievements and areas of application, and highlights key ... 

10 Article abstracts with full text online: Modeling biological systems: a unified approach Q 
Manuj Darbari, Vipin Saxena 

September 2005 ACM SIGSOFT Software Engineering Notes, Volume 30 issue 5 
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Publisher: ACM Press 

Full text available: ^ pdf(355.12 KB) Additional Information: full citation , abstract , references , index terms 

In this paper we demonstrate the relevance of abstraction, reuse, objects, classes, 
component and inheritance hierarchies, multiplicity, visual modeling, and other current 
software development best practices. We show how It is possible to start with a direct 
diagrammatic representation of a biological structure such as a cell, using terminology 
familiar to biologists, and by following a process of gradually adding more and more 
detail, arrive at a system with structure and behavior of arbitrary ... 

Keywords: UML, agent-based modeling, cell simulation 



n Components: A tool for component based design of embedded software 

David Urting, Yolande Berbers, Stefan Van Baelen, Tom Holvoet, Yves Vandewoude, Peter 
Rigole 

February 2002 Proceedings of the Fortieth International Conference on Tools Pacific: 
Objects for internet, mobile and embedded applications CRPIT '02 

Publisher: Australian Computer Society. Inc. 

Full text available- fS pdffl. 12 MB) Additional Information: full citation , abstract , references , citings, index 
' ^ ' terms 

In this paper a new approach for building embedded applications is presented. The 
approach is based on the composition of reusable components with the addition of a 
contract principle for modelling non-functional constraints. Non-functional constraints are 
an important aspect of embedded systems, and this is why they are modelled separately. 
As such, the component view presented here differs from traditional component based 
views, where focus Is laid on the functional part. The Ideas discussed i ... 

Keywords: component, contract, embedded, real-time, tool 



12 On utility-based selection of architecture-modellin g concep ts 
H. A. Proper, A. A. Verrijn-Stuart, S. J. B. A. Hoppenbrouwers 
January 2005 Proceedings of the 2nd Asia-Pacific conference on Conceptual 

modelling - Volume 43 APCCM '05 
Publisher: Australian Computer Society, Inc. 

Full text available: ^ pdf(229.48 KB) Additional Information: full citation , abstract , references , index terms 

In this position paper we are concerned with the principles underlying the utility of 
modelling concepts, in particular in the context of architecture modelling. First, some 
basic concepts are discussed, in particular the relation between information, language, 
and modelling. Our primary area of application is the modelling of enterprise architectures 
and information system architectures, where the selection of the concepts used to model 
different aspects very much depends on the specific concer ... 

Keywords: architecture modelling, modelling concepts, modelling languages 
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Jiff Matousek, Chi-Yuan Lo, William Steiger 

July 1992 Proceedings of the twenty-fourtii annual ACM symposium on Tlieory of 
computing STOC '92 

Publisher: ACM Press 

Full text available: « Ddf(679.04 KB) Additional Information: full citation , abstract, references , citings, index 
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Lo and Steiger resolved the complexity question for computing a planar ham-sandwich cut 
by giving an optimal linear-time algorithm. We show how to generalize the ideas to every 
fixed dimension d > 2 by describing an algorithm that computes a ham -sandwich cut in 
Rd in time 0(nd-l-a(d)), for some a( 

14 Classics in software engineering Q 
January 1979 Divisible Book 
Publisher: Yourdon Press 

Full text available: ^ pdf(22>45 MB) Additional Information: full citation , cited by . index terms 



15 Software Blueprints 

Patrica H. Baucom 

December 1978 Proceedings of the 1978 annual conference ACM 78 
Publisher: ACM Press 

Full text available- 1 ?lpdf(541.95 KB) Additional Information: full citation , abstract , refer ences, citing s, index 

terms 

The process of developing a software product is analogous to the process by which a 
building concept becomes a reality. Reliability is insured in both processes by validation of 
the design with a blueprint prior to construction. Just as the architect relies on blueprints 
to validate his plans for construction of a building, so may the software developer rely on 
software blueprints to validate his plans for construction of a software product. The 
sequence of steps in software development inc ... 

Keywords: Chapin charts, Iterative refinement, Nassi-Shneiderman charts, Program 
design language. Software design, Stepwise refinement, Structured code. Structured 
design. Structured flowcharts, Structured programming, Structured programming team, 
Structured wall<through 



An engineering methodology for presenting software functional architecture Q 
Leighton R Scott 

May 1978 Proceedings of the 3rcl international conference on Software engineering 

ICSE '78 
Publisher: IEEE Press 

Full text available: ^ pdf(324.20 KB) Additional Information: full citation , abstract , index terms 

A method of presenting software architecture has been developed which is useful for 
reviewing and documenting software designs. Diagrams showing all software levels as 
well as inputs and outputs are systematically developed to provide an understanding of 
the construction and operation of software systems and programs. The manner in which 
the software architecture is displayed provides an effective way for managers to 
understand and review software designs. 

Keywords; Design reviews, Software architecture. Software engineering blueprints, 
Software management, Structured design 
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To increase speed and reliability of operation, multiple computers are replacing 
uniprocessors and wired-logic controllers in modern robots and industrial control systems. 
However, performance increases are not attained by such hardware alone. The operating 
software controlling the robots or control systems must exploit the possible parallelism of 
various control tasks in order to perform the necessary computations within given real- 
time and reliability constraints. Such so ... 

18 A blueprint for automatic Boolean query processing 
^ G. Salton ^ 

^ September 1982 ACM SIGIR Forum, volume 17 issue 2 
Publisher: ACM Press 

Full text available: ^ pdf(932.06 KB ) Additional Information: full citation , abstract , references , citings 

Conventional information retrieval systems use Boolean query formulations and inverted 
file technologies for search and retrieval purposes. The need to construct complex 
Boolean queries in order to obtain the benefit of the existing retrieval operations 
constitutes a substantial burden for the users. In most environments trained search 
intermediaries are used to facilitate the communication between system and user.In this 
note a new Boolean retrieval environment is outlined in which the queries ... 

19 The measurement of information systems effectiveness: evaluating a measuring 
^ instrument 

^ Judy E. Scott 

February 1995 ACM SIGMIS Database, Volume 26 issue i 

Publisher: ACM Press 

Full text available: ^ pdfn.49 MB) Additional Information: full citation , abstract , citing s, index terms 

Information system effectiveness is an important phenomenon for both researchers and 
practitioners. Despite widespread interest, and the importance of the uses, there have 
been no efforts to validate Computerworld's Premier 100 rankings of information system 
effectiveness. This paper uses structural equation modeling in an attempt to validate the 
measuring instrument used to derive the Computerworld rankings.Alternative models for 
the measuring instrument are proposed. Using a refl ... 

20 An empirical evaluation of desi gn rati ona le doc uments 

Laurent Karsenty 

April 1996 Proceedings of the SIGCHI conference on Human factors in computing 
systems: common ground CHI '96 

Publisher: ACM Press 

Full text available: ^ pdf(972.Q4 KB) . » 
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Texture Mapping 3D Models of Real-World Scenes 
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Texture mapping has become a popular tool in the computer graphics industry in 
the last few years because it is an easy way to achieve a high degree of realism in 
computer-generated imagery with very little effort. Over the last decade, texture- 
mapping techniques have advanced to the point where it is possible to generate 
real-time perspective simulations of real-world areas by texture mapping every 
object surface with texture from photographic images of these real-world areas. 
The techniques for generating such perspective transformations are variations on 
traditional texture mapping that in some circles have become known as the 
Image Perspective Transformation or IPT technology. This article first presents a 
background survey of traditional texture mapping. It then continues with a 
description of the texture-mapping variations that achieve these perspective 
transformations of photographic images of real-world scenes. The style of the 
presentation is that of a resource survey rather than an in-depth analysis. 

Categories and Subject Descriptors: 1.3.3 [Computer Graphics]: Picture/Image 
Generation — antialiasing; 1.3.5 [Computer Graphics]: Computational Geometry 
and Object Modeling-c uri;e, surface^ solid, and object representations; hierarchy 
and geometry transformations; 1.3.7 [Computer Graphics}: Three-Dimensional 
Graphics and Realism-co/or, shading, shadowing, and texture; hidden line / surface 
remoual; raytracing 

General Terms: Algorithms 

Additional Key Words and Phrases: Anti-aliasing, height field, homogeneous 
coordinates, image perspective transformation, image warping, multiresolution 
data, perspective projection, polygons, ray tracing, real-time scene generation, 
rectification, registration, texture mapping, visual simulators, voxels 

INTRODUCTION decade in the creation of ever more real- 

istic looking images. Techniques have 
The computer graphics industry has been developed to generate illumina- 
made dramatic advances during the last tion-based shading effects, shadows, re- 
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flection, refraction, motion blur, lens de- 
focus, and such natural phenomena as 
sky, clouds, haze, fire, trees, grass, and 
water waves. Some of these techniques 
have even been incorporated into flight 
and vehicle simulators such as those 
built by General Electric Controls & 
Guidance Systems Division (now Lock- 
heed-Martin Co. Orlando, FL), Evans & 
Sutherland, and others. Unfortunately, 
many of these techniques are too com- 
putationally intensive for such real- 
time applications. For a survey of syn- 
thetic scene generation techniques, see, 
for example, Magnenat-Thalmann and 
Thalmann [1987]. 

Today, more demands are being 
placed upon visual simulators to 
achieve yet a higher level of realism 
[Fischetti and Truxal 1985; Tucker 
1984]. In particular, mission planning 
and rehearsal systems are now striving 
for truly faithful representations so that 
ground troops can become intimately fa- 
miliar with important regions of the 
world [Geer and Dixon 1988]. This level 
of detail and realism is possible only by 
using digitized photographs of the areas 
of interest. 

The technology is available today. Ex- 
amples have already been presented to 
the general public in the form of movies 
at conferences and on public television, 
and the like. The Jet Propulsion Labo- 
ratory has made several of these mov- 
ies. Two of these are fly-overs, called 
"LA— The Movie'' [Hussey et al. 1987] 
and "Mars— The Movie," [Hall 1989], 
which were presented at SIGGRAPH 
conferences and on PBS television. A 
third one depicts travel along the San 
Andreas fault in California [Stanfill 
1991]. It has been shown as part of the 
film, "The Blue Planet," at IMAX® and 
OMNIMAX® theaters worldwide. A dif- 
ferent fly-around of California was gen- 
erated by ESL (now TRW, Sunnyvale, 
CA) and was shown at the 1988 World's 
Fair in Australia.^ Another example is 
the video of the Calgary area that was 



^ Produced under the technical leadership of John 
Thomas at ESL. 



created by The Analytical Sciences Cor- 
poration for ABC News and Sports [Mc- 
Millan 1988; Gelberg et al. 1988]. It was 
shown during the 1988 Winter Olym- 
pics. Also a fly-around of Mt. Everest 
was created by Visual Information 
Technologies (VITec, now Connectware 
Inc.). It was shown as part of the 1991 
NOVA program, "Taller Than Everest?" 
Finally, Volotta Interactive Video has 
put together a system, called the Mars 
Navigator [Volotta 1991], to present low 
altitude, computer-generated fiy-bys of 
the surface of Mars prepared by Apple 
Computer [Hughes 1991] from Viking 
orbiter imagery. This system is on dis- 
play at the Tech Museum of Innovation 
in San Jose. 

The techniques used to make such 
images and movies are variations of the 
computer gi'aphics technique called tex- 
ture mapping. They have also come to 
be known by such names as image per- 
spective transformations (IPT), 2.5D 
image transformations, and perspective 
photo mapping. The first name comes 
from its origins in image processing, 
remote sensing, and photogrammetry. 
The second name reflects the fact that 
the transformation is between two-di- 
mensional (2D) and three-dimensional 
(3D); that is, the resulting views ac- 
count for perspective depth, but are not 
truly three-dimensional in the sense of 
full stereoscopic viewing. The third 
name has been coined to show its exten- 
sion from traditional texture mapping. 

Some of the salient challenges that 
must be met to achieve these images 
and movies and to create effective vi- 
sual simulations include: 

(1) How to create high resolution mod- 
els from heterogeneous input 
sources such as magazine pictures, 
direct photography, digital imagery, 
artists renderings, and the like. 

(2) How to align or register the imagery 
with the geometry models and 
smoothly blend different models 
from different sources so that noth- 
ing stands out unnaturally. 
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Table 1. Comparison of Traditional Texture Mapping with IPT 



Traditional Texture Mapping 

• Texture maps are frequently synthetic 

• Relatively few small texture maps are used 

• Texture maps are frequently repeated on 
multiple objects or faces 

• Texture maps are typically face-on views 

• Textures are often arbitrarily mapped onto 
objects without concern for distortions 

• Alignment of texture maps with the object 
faces is generally done manually 

• 3-D object models are typically built 
independently from the texture maps 

• Polygon object models and rendering 
approaches are typically used 



(3) How to deal efficiently with very 
large databases to keep rendering 
times down while maintaining real- 
ism. 

— Geometry models may include 
both natural terrain and urban 
features, where the former may 
contain millions of (regularly or 
irregularly spaced) elevation val- 
ues. 

— Many large, oblique, high-resolu- 
tion images may be needed to 
cover the vast expanse of natural 

terrain. 

(4) How to correct source imagery dis- 
tortions so that resulting output 
views are faithful. 

— Source images are often oblique 
views. 

— Source images are formed by vari- 
ous camera and sensor types often 
with nonstandard geometries. 

(5) How to prewarp the output images 
so they project naturally onto 
domes, pancake windows, and the 
like. 

(6) How to perform these myriad tasks 
fast enough for real-time or high 
speed nonreal-time display. 

For the purposes of discussion in this 
article, we refer to the texture-mapping 
variations that attempt to solve some or 



Image Perspective Transformation 

• Texture maps are actual photographs or 
remote sensing images of real-world areas 

• Many large images are used as texture maps 

• Textures are unique per object face 

• Texture maps are typically oblique views 

• Photogrammetry techniques are often used to 
project the textures onto object faces to correct 
for camera acquisition geometry 

• Photogrammetric/remote sensing techniques 
are frequently used to automatically align the 
textures with the object faces 

• Photogi'ammetry techniques are frequently 
used to build 3-D object models of terrain and 
urban structures from the imagery 

• Height field, voxel and polygon object models 
and rendering approaches are all used 



all of these challenges as the image 
perspective transformation. Although 
the distinction between the IPT texture- 
mapping variation and traditional tex- 
ture mapping may be subtle, these dif- 
ferences are worth noting. Some people 
may argue that they are fundamentally 
just differences in terminology arising 
from the perspectives of the disciplines 
that originated them; namely, image 
processing, remote sensing, and photo- 
grammetry in the former case and com- 
puter graphics in the latter case. To a 
large extent this is true, especially with 
regard to the basic objectives, namely, 
to transfer image texture onto the sur- 
faces of 3D models. Moreover, the two 
technologies are rapidly converging. 
Nevertheless, some of the conceptual 
differences are worth emphasizing as 
well as some of the novel algorithmic 
approaches that have been developed 
that are quite different from the basic 
polygon-based approach used in tradi- 
tional texture mapping. Consequently, 
some of the important differences be- 
tween the image perspective transfor- 
mation and the traditional texture-map- 
ping technique are summarized in the 
following and listed in Table 1. A math- 
ematical treatise of traditional and pro- 
jective texture mapping is presented in 
Weinhaus and Devich [1997]. 
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With traditional texture-mapping 
techniques, texture patterns are mathe- 
matically transferred first onto the sur- 
faces of 3D models that are used to 
represent either fictitious or real-world 
scenes. Then, from this intermediate 
format, they are perspectively projected 
onto the output viewing plane. Usually, 
these texture patterns are computer- 
synthesized ones representing generic 
surface materials such as grass or wood, 
or they are digitized photographs of rep- 
resentative materials viewed face-on 
such as brick or concrete. Sometimes, 
they are even photographs of real-world 
scenes used to represent backdrops, 
views out of windows, reflection pat- 
terns of the surrounding scene, or pic- 
tures to be incorporated on interior 
walls of rooms. These texture patterns 
typically are mapped orthogonally onto 
planar object faces with a simple affine 
transformation. However, for certain 
applications, they either may be pre- 
warped or treated as a parametric rep- 
resentation of a coordinate system such 
as cylindrical or spherical in order to 
wrap them about an object conforming 
to the same geometry. Texture tiling 
[Dungan et al. 1978], cell texturing 
[Economy and Bunker 1984; Bunker et 
al. 1984], decal mapping [Barr 1984], 
reflection mapping [Blinn and Newell 
1976], and environment mapping 
[Green 1986] are some of the specific 
names used to describe variations of the 
texture mapping technique. 

With the IPT technique, the textures 
also come from digitized photographs; 
however, in this case, the texture pat- 
terns are the complete and often oblique 
photographs of real-world areas that 
are represented by the 3D models. The 
objective is to start from a set of source 
photographs and create new views of 
the real-world areas from arbitrary van- 
tage (eye) points and, if possible, to do 
so in real-time (i.e., 30 or 60 frames per 
second). 

For the traditional texture-mapping 
case, one generally uses only a limited 
number of relatively small texture pat- 
terns. Usually, one texture pattern is 



defined to span the space of the surface 
in question, since there is often a simple 
correspondence between the polygonal 
bounds of the texture pattern and the 
polygonal shape of the surface. Some- 
times, however, a single texture pattern 
may be repeated to fill out the coverage 
of a surface or the same pattern may be 
assigned to many neighboring surfaces. 
In this context, a surface can be a pla- 
nar or nonplanar polygon, a nonplanar 
patch, or even the complete surface of a 
quadratic object such as a cylinder or 
sphere. Thus, for example, it may be 
that a texture pattern is arbitrarily 
stretched using a "rubber sheet" (affine 
or bilinear) transformation onto a pla- 
nar quadrilateral or parametrically 
wrapped about the surface of a cylindri- 
cally symmetric or spherical object. As a 
consequence of the rather arbitrary 
mapping onto nonplanar objects, the re- 
sulting appearance of the texture cover- 
ing for the surface may be distorted. 
Nevertheless, this may not be notice- 
able due to the lack of real-world corre- 
spondence between the texture and the 
model. 

For the IPT case, many large photo- 
graphic texture images may be needed 
to cover large expanses of territory. 
Also, different portions of one or more 
photographs must be mapped onto the 
numerous surfaces that form the com- 
plex 3D models of the real-world area. 
This photographic information must 
align with the 3D models without dis- 
tortion and with minimal manual inter- 
vention. This is often accomplished as a 
preprocessing step using photogramme- 
try and remote sensing registration 
techniques. More optimally, the texture- 
mapping process should automatically 
take into account the acquisition geom- 
etry of the system that took the photo- 
graphs. For pictures acquired by a 
frame (snap-shot) camera system, this 
mapping must correct for perspective 
distortion inherent in the source imag- 
ery. This distortion causes obliquely 
viewed rectangles to appear as quadri- 
laterals. For this camera system, two 
perspective transformations are in- 
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volved: one to map the texture onto the 
3D models and one to project this inter- 
mediate result onto the output viewing 
plane. Photogrammetry equations are 
often used to represent both of these 
perspective transformations. 

The objectives of this article are 
threefold: to present the historical back- 
ground for the image perspective trans- 
formation technique starting with its 
origin in texture mapping, to introduce 
its mathematical foundation (a detailed 
mathematical treatise of texture map- 
ping can be found in Weinhaus and 
Devich [1997]), and to identify the vari- 
ous approaches that have been taken, 
giving credit to the many people and 
organizations that have contributed to 
it. The emphasis of this article is pri- 
marily on the geometry of the transfor- 
mation; however, the issue of rendering 
quality is also addressed. Our approach 
to this survey, therefore, is to provide 
only a simple description of the many 
techniques that have been developed, 
citing references so that the user subse- 
quently may review them in more depth 
and formulate his or her own judgment 
on their merits. Where appropriate, we 
point out the strengths and weaknesses 
of these approaches. However, it is not 
our intention to provide our biased se- 
lections of the "best" ones. Moreover, an 
algorithm that works well in a software 
environment may be quite inappropri- 
ate for hardware implementation. 

In the material that follows, Section 2 
first reviews the history of traditional 
texture mapping. It then describes the 
mathematical foundation and the devel- 
opment of texture-mapping approaches. 
Section 3 starts with a similar historical 
review for the image perspective trans- 
formation technology. It then reviews 
the IPT-specific approaches that have 
been developed, subdividing them into 
three categories according to the way 
the 3D model is represented: height 
fields, voxels, and polygons/patches. In 
Section 4, nonreal-time and real-time 
systems are discussed with emphasis on 
the technology arising from the flight 
simulator community. 



2. TEXTURE MAPPING 

2.1 Texture Mapping Background 

The pioneering work in texture map- 
ping is generally attributed to CatmuU 
[1975]. He was probably the first to 
demonstrate the mapping of a (brick) 
texture pattern onto arbitrary planar 
and cylindrical surfaces. Blinn and 
Newell [1976] then took the concept one 
step further by mapping photographic 
textures onto a bicubic patch represen- 
tation of the now famous teapot. Aoki 
and Levine [1978] and Feibush et al. 
[1980] independently used the tech- 
nique to render synthetic textures onto 
the faces of 3D models of houses. Fei- 
bush and Greenberg [1980] also devel- 
oped a texture mapping system to be 
used for architectural design. Later, 
Economy and Bunker [1984] and Bun- 
ker et al. [1984] introduced the concept 
of computer-synthesized "cell" textures 
to cover ground terrain in visual flight 
simulators, and Dungan et al. [1978] 
applied actual photographic texture pat- 
terns to cover the ground terrain. In the 
latter case, they acquired vertical pho- 
tographs of representative tree-covered 
terrain from low-flying aircraft and pho- 
tographs of grass textures from model 
boards. These texture patterns were 
then mapped onto 3D polygonal terrain 
elevation models in their simulated 
scenes. These authors were among the 
first to publish examples using textures 
with multiple levels of resolution as a 
method of anti-aliasing. Later, the sci- 
entists at General Electric [Economy 
and Bunker 1984; Bunker et al. 1984] 
and Honeywell [Scott 1983; Erickson et 
al. 1984] developed applications of tex- 
ture mapping of photographs of individ- 
ual trees onto planar "billboards" and 
onto multifaceted 3D models of individ- 
ual trees. Transparency effects were in- 
troduced to make the texture regions 
around the trees and between the leaves 
invisible. Bunker et al. [1984] described 
similar translucency techniques for in- 
serting texture patterns of cloud layers 
into simulated scenes. 
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Figure 1. Texture-mapping geometry. 



The geometry of texture mapping is 
depicted in Figure 1. Two transforma- 
tions are involved. One transformation 
is between object space and texture (in- 
put) space and the other is between 
object space and screen (output) space. 
The object space is characterized by the 
equation of each surface of the 3D 
model. The object-texture transforma- 
tion may be as simple as an affine or 
bilinear transformation when the tex- 
ture is to be mapped orthogonally onto a 
planar quadrilateral. Alternately, it 
may be a parametric transformation 
when the texture coordinates are used 
to represent nonCartesian coordinates, 
such as cylindrical or spherical, and the 
textures are to be "shrink-wrapped" 
about a 3D model with like symmetry. 
Bier and Sloan [1986] have presented a 
number of examples for this latter case 
and Heckbert [1989] has discussed at 
length various 2D transformations used 
to map textures onto planar and nonpla- 
nar polygons. The object-screen trans- 
formation is usually either an ortho- 
graphic projection (sometimes called a 
parallel plane projection) or a (central) 
perspective projection. The former in- 
volves imaging with parallel rays and 
exhibits uniform scale. The latter in- 
volves imaging with rays that converge 



to a focal point and exhibits scale that 
varies from foreground to background, 
that is, perspective foreshortening. Ta- 
ble 2 presents the form of some of these 
key transformation equations funda- 
mental to traditional texture mapping 
and the image perspective transforma- 
tion variation. In these equations as 
relevant to the direction of the transfor- 
mation (forward or reverse), either (x, y, 
and optionally, z) or (X, Y, and option- 
ally Z) represents object space and the 
other represents either texture space 
(u, i;) or screen space (S, L). 

Often the object-texture and object- 
screen transformations are concate- 
nated in order to save computations. 
The resulting composite transformation 
usually can be formulated either as a 
forward (texture-to-screen) transforma- 
tion or as an inverse (screen-to-texture) 
transformation. Each has its own ad- 
vantages and disadvantages, which 
have been discussed at length by Heck- 
bert [1989] and by Wolberg [1990]. 

In general, anti-aliasing techniques 
for texture-mapping applications have 
been more elaborate than for the more 
traditional coloring and/or illumination- 
based rendering techniques in order to 
accommodate the combination of fine 
detail in photographic imagery and spa- 
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Name 



Properties 



Form 



AFFINE 
(LINEAR) 



BILINEAR 



QUADRATIC 



CUBIC 



PROJECTION 

• Perspective 

• Orthographic 



• Translation, Scale, Rotation 
And Skew 

• Preserves Straight Lines, 
Parallelism And Scale 
Homogeneity 

• Preserves Straightness Of 
Horizontal & Vertical 
Lines 

• Maps Other Straight Lines 
Into Hyperbolic Curves 

• Maps Straight Lines Into 
Quadratic Curves 

• Maps Straight Lines Into 
Cubic Curves 



• Preserves Straight Lines 



x^Ao-^AiX + A2Y 



x = Ao+ A,X + A2Y + A^ + A,X^ + A^Y^ 

y = Bo + B,X + B2Y + B3XY + B^X^ + BsY^ 
jc = Ao + AiX + A2Y + AgXy + A,X' + AsV^ 
+ AcX'Y 4- AjXY" + A^' -f A,Y^ 

y = Bo + B,X 4- B2Y + B^XY + B^X' + B^Y^ 
+ B^'Y + B^XY' + B^ + B^Y^ 
Matrices 
P= projection 
R=rotation 
T= translation 



^wx\ 




(x\ 


wy 


= PRT 


Y 


LVZ 




Z 






\ll 



where 



w is eliminated by dividing the 4th 
component into the other three 



f = focal length for the perspective case 
(1//) == 0 and «; ~ 1 for the orthographic 
case 




* Shows the form of some of the transformation equations fundamental to traditional texture mapping 
and the image perspective transformation variation. In these equations, as relevant to the direction of 
the transformation (forward or reverse), either ix, y, and optionally, z) or {X, Y, and optionally, Z) 
represent object space and the other represents either texture space (u, v) or screen space (S, L). 



tially varying scale when viewing in 
perspective. Some of the more sophisti- 
cated techniques that have been used to 
anti-alias texture-mapped images are 
cited here for completeness. They in- 
clude the following: trilinear interpola- 
tion on hierarchical resolution texture 
patterns (called pyramids [Tanimoto 
and Pavhdis 1975; Burt 1981] and MIP 
maps [Williams 19831), elliptically 
weighted averaging (EWA) [Greene and 
Heckbert 1986], spatially variant filter- 
ing [Fournier and Fiume 1988], 
summed area tables [Crow 1984; Glass- 
ner 1986], repeated integration [Heck- 



bert 1986a], clamping [Norton et al. 
1982], super-resolution sampling, sto- 
chastic and jittered sampling [Cook 
1986; Dippe 1985], adaptive sampling 
[Painter and Sloane 1989], A-buffering 
[Carpenter 1984], accumulation-buffer- 
ing [Haeberli and Akeley 1990], and 
spatial transformation lookup tables 
[Walterman and Weinhaus 1991]. Heck- 
bert [1986b] has presented a review and 
comparison of many of these anti-alias- 
ing techniques. Some of these are dis- 
cussed later in the context of the de- 
scription of a particular texture 
mapping or IPT algorithm. 
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2.2 Texture Mapping Approaches 

The fundamental, nontexture-mapped 
rendering technique used by most 3D 
graphics systems typically projects the 
vertices of each primitive (usually pla- 
nar polygons) from world coordinates 
into screen coordinates using a perspec- 
tive projection equation as shown in Ta- 
ble 2. It then linearly interpolates the 
color (and perspective depth) at each 
edge and interior pixel of this projected 
primitive. Usually, an algorithm, such 
as the digital differential analyzer 
(DDA) [Swanson and Thayer 1986; 
Pineda 1988] or forward difference tech- 
nique [Lien et al. 1987; Shantz and Lien 
1987] is used to do the interpolation 
from the color and perspective depth 
values stored for each vertex. Such algo- 
rithms are very efficient, since they ex- 
ploit incremental computations along 
each scan-line of the projected primi- 
tive. The well-known Gouraud shading 
technique is an example of this type of 
interpolation where the shading infor- 
mation associated with the vertices is 
determined by an illumination model in 
conjunction with the vertex normals. 
The Phong shading technique is similar, 
but interpolates the vertex normals be- 
fore determining the shading value at 
each interior pixel. See, for example, 
Rogers [1985] or Foley et al, [1993] for 
more details about these two specific 
shading techniques. 

As pointed out by Heckbert and IMore- 
ton [1991] and by Blinn [1992], a naive 
approach to texture mapping would 
store the corresponding texture coordi- 
nates for each vertex and try to inter- 
polate these coordinates in a similar 
manner. The interpolated texture coor- 
dinates would then be used to index into 
the texture image to extract the appro- 
priate color values. Unfortunately, this 
procedure introduces distortions that 
may be noticeable, because the tech- 
nique approximates the perspective 
projection with a linear transformation. 
Moreover, polygons are usually subdi- 
vided into combinations of triangles and 
quadrilaterals that have one or two 



edges aligned along the scan direction, 
respectively. Each triangle or quadrilat- 
eral ultimately will use a different lin: 
ear transformation. This may create 
sharp bends or folds in transformed lin- 
ear features as they pass from one sub- 
polygon to another. A demonstration of 
this effect is presented in Figure 2(b) 
where the quadrilateral has been subdi- 
vided along one of its diagonals into two 
triangles. Since the other diagonal must 
pass through the midpoint of the first 
diagonal, it will be transformed into two 
joined line segments, each with differ- 
ent slopes. Figure 2(a) shows the origi- 
nal texture map. 

To avoid such discontinuities for 
quadrilaterals, one might try using a 
bilinear transformation rather than a 
linear one to interpolate the texture co- 
ordinates. Unfortunately, the bilinear 
transformation only preserves straight 
lines originally parallel to the coordi- 
nate axes of the texture map. Conse- 
quently, the diagonals will appear 
curved in the resulting interpolated 
polygon. A proper perspective transfor- 
mation will preserve all straight lines. 
Demonstrations of these characteristics 
are presented in Figures 2(c) and 2(d). 
These three transformations also affect 
scale differently. The linear and bilin- 
ear transformations presei^e scale 
across the texture pattern better than 
the perspective transformation. This ef- 
fect can be obsei^ved in the figures by 
examining the spacing of the grid pat- 
tern, especially between those lines that 
correspond to the horizontal ones in the 
original texture image. In Figure 2(d), 
this spacing progressively increases 
from top to bottom, whereas, in Figures 
2(b) and 2(c), it remains constant. 

Note that the linear approximation 
gets progressively better for smaller 
polygons. Thus, CatmulVs [1975] ren- 
dering algorithm, which adaptively sub- 
divides surfaces into pixel-sized mi- 
cropolygons, is well suited to handle 
this problem. Following this premise, 
Oka et al. [1987] used a local linear 
approximation to perform texture map- 
ping of photographs of people onto 
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Figure 2. Comparison of several texture-mapping transformations: (a) original texture map; (b) after 
linear transformation; (c) after bilinear transformation; (d) after perspective transformation. 



quadrilateral mesh models of human 
faces. They were then able to modify 
facial expressions by interactively ma- 
nipulating the mesh. 

Kirk and Voorhies [1990] have dem- 
onstrated the use of a quadratic interpo- 
lation function rather than a linear one 
to approximate the perspective transfor- 
mation for texture-mapping applica- 
tions. Other examples of the use of qua- 
dratic as well as cubic approximations 



for texture mapping in perspective have 
been presented by Wolberg [1990]. How- 
ever, polygon subdivision may still be 

necessary to achieve acceptable results. 

The proper mathematical basis for 
texture mapping onto planar polygons 
has been described in detail by Aoki and 
Levine [1978], Gangnet et al. [1982], 
Perny et al. [1982], Heckbert [1989; 
1983], Heckbert and Moreton [1991], 
and Blinn [1992] for the case in which 
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the texture is to be mapped with an 
affine transformation and where the 
output viewing transformation is a per- 
spective projection. These two transfor- 
mations can be concatenated into a sin- 
gle fractional linear form that is 
applicable in either direction. Thus, it 
can be used to describe either forward 
mapping from texture space to screen 
space or inverse mapping from screen 
space to texture space. For the inverse 
mapping case, this transformation can 
be expressed as follows. 

Po + ^iS + P2L 

u == 

(1) 

Rq + RiS + R2L 

where the pixels in the texture image 
are represented by the (u, v) coordi- 
nates and those in the output image are 
represented by the (S, L) coordinates. 
Note that without any loss of generality, 
the zeroth order coefficient in the de- 
nominator can be set to unity; that is, 
the set of equations can be normalized 
by dividing both numerators and the 
common denominator by this term. 
From Equation (1), it is easy to see that 
the two numerator terms and the one 
denominator term are the proper ex- 
pressions to interpolate, because each of 
them is a linear polynomial in S and L, 
This is often done simply by increment- 
ing them by P^, Qy, and R^, respec- 
tively, along a given scan-line. Since the 
texture coordinates are composed of ra- 
tios of these linear polynomials, they 
are not good choices for the linear inter- 
polation technique. Blinn [1992] refers 
to interpolation with this fractional lin- 
ear form as hyperbolic interpolation. 

Equation (1) is generally used for tex- 
ture mapping onto planar rectangles, 
because it requires a minimum of four 
pairs of corresponding texture and 
screen coordinates to solve for the eight 
unknown coefficients. The four vertices 
of the rectangle projected into screen 
space in the form of a planar quadrilat- 



eral and the four corresponding bound- 
ing corners of the rectangular texture 
image provide a convenient set. The so- 
lution can be achieved by multiplying 
through by the denominator and recast- 
ing the equation in the following form: 

{l)Po'^{S^P^'^{L^P2-{SiU^R, 

- {LiUi)R2 = Ui 

(2) 

(l)Qo + (S,:)Q, + (L,)Qo-(S,f;,)7?, 

- {LiVi)R2 = Vi, 

where the zeroth order coefficient in the 
denominator has been set to unity. Sub- 
stitution of each corresponding set of 
coordinates (i = 1 to 4) into this normal- 
ized pair of equations provides a set of 
eight linear equations in eight un- 
knowns that can be solved, for example, 
by the unit„square-to-quadrilateral 
technique described by Heckbert [1989]. 
As Heckbert also points out, this ap- 
proach can be extended to handle the 
case of mapping a quadrilateral region 
of texture onto a planar rectangle, since 
the solution and its inverse can be 
concatenated easily to form a quadri- 
lateral-to-unit_square-to-quadrilateral 
transformation. 

Hourcade and Nicolas [1983] have 
taken the development a step further. 
They used a bilinear transformation 
rather than an affine one to map the 
texture onto the polygon and then 
viewed it in perspective. The use of the 
bilinear form, which includes a 
crossterm, permits a rectangular tex- 
ture image to be mapped onto more 
general planar or even nonplanar quad- 
rilaterals, neither of which are compati- 
ble with the affine transformation. For 
the forward (texture-to-screen) mapping 
case, Hourcade showed that the compos- 
ite transformation can be expressed as 
the ratio of polynomials up to and in- 
cluding the (uv) crossterm. However, 
for the inverse (screen-to-texture) trans- 
formation, a quadratic equation must be 
solved for each screen pixel. (The qua- 
dratic characteristic of the inverse bilin- 



ACM Computing Surveys, Vol. 29, No. 4, December 1997 



Texture Mapping 3D Models • 335 



ear transformation also has been de- 
scribed by Heckbert [1989] and by 
Wolberg [1990].) 

Tlie preceding analysis follows what 
is commonly called a single-pass, 2D 
approach. Catmull and Smith [1980] 
and Smith [1987] have taken a different 
tack. They developed a flexible and effi- 
cient forward mapping, two-pass, ID 
approach. In the first pass, the texture 
pattern is warped only in one dimension 
(horizontally or vertically) to form an 
intermediate result. In the second pass, 
the intermediate result is warped only 
in the orthogonal dimension. These au- 
thors presented equations for texture 
mapping onto planar and bilinear quad- 
rilaterals, biquadratic and bicubic 
patches, and superquadrics. For the 
case of texture mapping onto a planar 
rectangle and perspective viewing, they 
showed that the two ID transforma- 
tions are also fractional linear in form. 
For the other cases, ratios of higher- 
order polynomials are involved. 

In the review previously presented, it 
was generally assumed that the com- 
plete rectangular texture pattern was 
mapped one-to-one onto a four-sided 
polygon or patch. However, this condi- 
tion may be relaxed by using projections 
to determine the texture coordinates as 
well as the screen coordinates. In other 
words, the affine object-texture trans- 
formation depicted in Figure 1 is re- 
placed with an orthographic projection. 
This will permit portions of the texture 
pattern to be mapped onto planar poly- 
gons with more than four sides. It is 
only necessary that the projection of the 
polygon onto the texture pattern does 
not extend outside the texture domain. 
In this way, a corresponding polygonal 
subsection of the texture pattern will be 
used to fill the interior of the polygon 
(i.e., a "cookie-cutter" technique). The 
texture coordinates corresponding to 
the projected polygon vertices can be 
predetermined and saved or computed 
at the time the coefficients of Equation 
(1) are evaluated. Then four of the ver- 
tices of the planar polygon can be pro- 



jected into screen space and Equation 
(2) solved for the coefficients. 

But what about triangles? One simple 
solution is to project the centroid of the 
triangle along with its vertices into tex- 
ture and screen space so that four con- 
jugate sets of points are available. Al- 
ternately, a more general solution can 
be developed. This is achieved by going 
back one step in the development of 
Equation (1), before the perspective di- 
vide, where the composite transforma- 
tion can be expressed in matrix form 
using homogeneous coordinates as 

lws\ 
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(3) 



Here, w and W are the corresponding 
third homogeneous coordinate compo- 
nents for any object space point pro- 
jected into texture and screen spaces, 
respectively. For more information 
about homogeneous coordinates, see, for 
example, Foley et al. [1993]. Note that 
w may be set to unity when the object- 
texture transformation is represented 
by an orthographic projection, so that 
u' = u and v' = v are true pixel coordi- 
nates in the texture pattern. Substitu- 
tion of any three sets of these corre- 
sponding coordinates into Equation (3) 
results in the readily solvable set of 
nine linear equations in eight un- 
knowns. 

This approach is easily generalized tn 
the case where a perspective projection 
is used for the object-texture transfor- 
mation. In this case, w will take on 
values other than unity. The disadvan- 
tage is that three floating-point texture 
(and screen) homogeneous coordinate 
components are required for each ver- 
tex. This would increase the number of 
bits of vertex information that would 
have to be carried with the 3D models, 
if one desired to precompute and store 
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them rather than compute them only 
when needed. 

An important characterization of 
Equation (1) has been described by 
Fuchs et al. [1985]. They showed that 
for texture mapping with an affine ob- 
ject-texture transformation the denomi- 
nator term in Equation (1) is equivalent 
to inverse depth in the viewing or eye 
coordinate system, that is, Rq + R^S + 
R2L = (1/Z^,). Here is the coordinate 
component in the eye coordinate system 
along the look direction. This implies 
that inverse depth should be interpo- 
lated and used in depth buffer algo- 
rithms rather than simple depth. The 
numerator terms can then be equated to 
the product of inverse depth with the 
texture coordinate components, namely, 
(w/Z,) = Po + P,S + P2L and (i;/ZJ = 
Qo + Qi*5 + Q'l^^ ^i^d these products 
can be precomputed for each vertex of 
the polygon. Equation (1) can be evalu- 
ated subsequently for each screen pixel 
within a polygon simply by interpolat- 
ing these precomputed products {u/ZJ 
and (f/Zg), representing the numerator 
terms, and then dividing the interpo- 
lated results by the interpolated denom- 
inator term (1/Zg). The advantage of 
this approach is that the coefficients of 
the fractional linear equation never 
need to be evaluated. Note that the 
equivalence between the denominator 
term and inverse depth is not justified 
for the case in which the object-texture 
transformation is a perspective projec- 
tion. 

An analogous approach, but ex- 
pressed using homogeneous coordinates, 
was first described by Heckbert and 
Moreton [1991] and later by Blinn 
[1992]. Heckbert and IVIoreton showed 
that each of the terms (u/w), (f/cu), and 
(l/a>) are linear in S and L and so 
should be interpolated rather than sim- 
ply u and V. Here co is the fourth homo- 
geneous coordinate in the transforma- 
tion of eye coordinates (X,,, Y^, Z,,, 1) to 
homogeneous eye coordinates (wX,,, 
wY^^, (oZ^, 0)). Then, dividing the first 
two interpolated terms pixel by pixel by 
the fourth interpolated term recovers 



the interpolated texture coordinates u 
and y. The equivalence of this approach 
with the previous one is apparent, since 
(o = (ZJf) [Foley et al. 1993], where /is 
the distance between the eye point and 
picture plane in the viewing pyramid. 

Heckbert and IMoreton [1991] took 
this approach a step further and showed 
that the correct terms to interpolate 
when the object-texture transformation 
is a perspective projection rather than 
an affine transformation are (ucj'/cj), 
{vo)'/(x})y and (wVco), where co' is analo- 
gous to (i>, but is for the texture projec- 
tion coordinate system. Segal et al, 
[1992] have described an implementa- 
tion of this projective texture-mapping 
approach to demonstrate shadows, spot 
lighting, and slide projection effects in 
computer-generated images. 

One can list numerous past and 
present research and commercial com- 
puter graphics systems as well as the 
commercial flight simulators mentioned 
earlier that use or have used texture 
mapping to a limited extent. A few of 
these are hardware and software sys- 
tems developed by Hewlett-Packard, 
Pixar (Reyes Image Rendering Architec- 
ture [Cook et al. 1987] and RenderMan 
software [Upstill 1990]), Schlumberger 
Palo Alto Research [Deering et al. 
1988], Silicon Graphics Inc. (SGI), Star- 
dent Computers (now Kubota Graphics), 
Sun IMicrosystems, and the University 
of North Carolina (Pixel-Planes [Fuchs 
et al. 1985]). These systems rely or re- 
lied primarily on traditional 3D graph- 
ics techniques such as Gouraud and 
Phong shading. By using special-pur- 
pose hardware, many are capable of 
rendering hundreds of thousands of 
polygons per second. All of these sys- 
tems can map textures onto surfaces, 
but most of them limit the amount of 
texture data that can be used so that 
rendering rates are not degraded by ei- 
ther disk-to-memory or memory-to-pro- 
cessor bandwidths. A few systems that 
use parallel processors store the same 
texture patterns in each processor's own 
local memory so that real-time render- 
ing rates can be maintained. Some of 
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the advanced simulators recently devel- 
oped by Evans & Sutherland, Lockheed- 
Martin Co. (Orlando, FL), Lockheed- 
Martin-Vought Systems (formerly LTV), 
and SGI have been designed specifically 
to address applications requiring high 
quality texture mapping of large quanti- 
ties of texture data. 

3. IMAGE PERSPECTIVE 
TRANSFORMATION 

3.1 Image Perspective Transformation 
Background 

The techniques used in the perspective 
transformation of images have evolved 
readily from those used to perform tex- 
ture mapping. The most important dif- 
ference between these two applications, 
however, is the need to handle larger, 
more numerous and frequently oblique 
photographic texture data in the former 
case. This implies that larger memory 
capacity, faster disks and higher band- 
width I/O channels are desirable. New 
algorithmic approaches also are needed 
to accommodate the increased load. 

Another important difference is the 
use of photogrammetry techniques to 
represent the texture imagery acquisi- 
tion geometry. The coUinearity condi- 
tion [Gosh 1979; Thompson 1966] in 
photogrammetry is the basis for charac- 
terizing the acquisition geometry. It 
states that a point on an object, its 
corresponding picture point, and the fo- 
cal or perspective center all lie on the 
same line. For a frame camera, this may 
be expressed in forward and inverse 
form as 
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where (x, y, f) represent the picture 
point, (AT, y, Z) represent the object 



point, {X^, y^, Z^.) represent the camera 
location, and R = is a 3D rotation 
matrix. As in the case of the perspective 
projection equation in Table 2, the third 
components here must be divided into 
the other two components in order to 
eliminate the unknown parameters jLt 
and A. In fact, Equation (4) is an alter- 
nate representation for perspective pro- 
jection. Imaging systems such as pan- 
oramic [Case 1967], strip [Case 1967], 
and multispectral scanners [Baker 
1977] (e.g., LANDSAT, SPOT, etc.) will 
have other mathematical forms. Never- 
theless, they all relate an object point to 
an image point. Having such equations 
permits the imagery to be "projected" 
onto the surfaces of the object models, 
automatically correcting for the imaging 
distortions inherent in these systems. 
IMoreover, they facilitate constructing 
the 3D object models, since a 3D point 
on an object may be identified by inter- 
secting two such "rays," one each pass- 
ing through corresponding feature 
points on two different images. Utiliza- 
tion of the coUinearity equations re- 
quires knowledge of the camera location 
and orientation parameters. These pa- 
rameters are deduced typically by non- 
linear least squares fit either for each 
image independently or simultaneously 
for a group of overlapping images. In 
the former case (called space resection), 
the least squares fit requires conjugate 
image and world "control" points. In the 
latter case (called block or bundle ad- 
justment), the least square fit uses a 
combination of the same along with con- 
jugate image "pass" points located on 
overlapping images. 

Rather than photogram metrically 
projectnig textures onto the surfaces of 
the terrain models, remote sensing reg- 
istration and resampling techniques are 
often used instead to preprocess the im- 
ages to align them with the terrain 
model. In effect, this preprocessing step 
removes the imaging distortion so as to 
rectify the imagery to a vertical view. 
Several different approaches have been 
used. One method performs a least 
squares fit of a set of arbitrarily spaced 
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control points using a low (typically sec- 
ond or third) order 2D polynomial trans- 
formation (see Table 2) to facilitate the 
registration and resampling operation 
globally across the image. A second 
method divides the image into a rectan- 
gular grid of control points and then 
uses a different bilinear transformation 
locally within each grid area to resam- 
ple the imagery [Rifman 1973; Bern- 
stein 1976; Van Wie and Stein 1977], A 
third method divides the image into a 
triangular mesh of control points and 
then uses a different affine transforma- 
tion locally within each triangle to re- 
sample the imagery [Goshtasby 1986, 
1987]. 

Another approach, called orthorectifi- 
cation [Hood et al. 1989; Friedman 
1981; Peled and Shmutter 1984], uses 
the collinearity equation and the terrain 
elevation model to correct for terrain 
relief distortion as well as the imaging 
distortion. In effect this technique is a 
special (reverse) case of traditional tex- 
ture mapping, where the object-texture 
transformation is a perspective projec- 
tion (for a frame camera system) and 
the object-screen transformation is a 
(vertical) orthographic projection (repre- 
sented by an affine transformation). It 
also differs in that the output image is 
typically much higher resolution (larg- 
er) than normal. It is essentially a case 
of a photogrammetry-based image per- 
spective transformation used to prepro- 
cess the imagery so that a simpler (af- 
fine) object-texture transformation can 
be used later in the final texture-map- 
ping operation. 

Subsequent sections describe the var- 
ious algorithmic approaches that have 
been developed for the IPT case. These 
approaches are categorized in several 
ways. One grouping characterizes them 
according to the type of hidden object 
removal (or visibility) technique that is 
used in the rendering (e.g., depth-prior- 
ity, depth-buffer, scan-line, ray tracing, 
etc.). For a general review of these visi- 
bility techniques, see, for example, Mag- 
nenat-Thalmann and Thalmann [1987], 
Rogers [1985], or Foley et al. [1993]. A 



second grouping characterizes them ac- 
cording to scene content, for example, 
urban versus rural (cultural versus nat- 
ural). A third grouping characterizes 
them by the class of 3D model primitive 
used to represent the setting, for exam- 
ple, a regular array of point features, 
voxels, polygons, or higher-order sur- 
faces and patches. 

Perhaps the earliest example of a per- 
spective transformation of an image of a 
real-world scene was that presented by 
Tanaka [1979]. He transformed a 
LANDSAT image of Mt. Fuji into a low 
altitude horizontal view. The type of 3D 
model that he used was a regular grid of 
elevation values interpolated from a 
contour map. 

In 1980, Lippman [1980] described 
the "IVIovie-lMap" concept of "surrogate 
travel" being developed at the IVIassa- 
chusetts Institute of Technology. In one 
study, frames of movie film were col- 
lected at 10 ft spacing while driving 
down the streets of Aspen, Colorado. 
Then selected portions of some of these 
frames corresponding to face-on views of 
building facades were mapped onto the 
faces of 3D models of the buildings. 
Distant mountains were also textured 
to add a sense of realism. Computer- 
synthesized images simulating driving 
down the streets as well as low altitude 
views were then generated in segments 
and stored on video disk for later play- 
back under joystick control. During 
playback, turns could only be initiated 
at selected locations such as street in- 
tersections, 

Devich and Weinhaus [1980, 1981a] 
at ESL (now TRW, Sunnyvale, CA), also 
in 1980, presented another example of a 
perspective transformation of an urban 
setting. Their example converted two 
oblique and one nadir-looking aerial 
photographs of downtown San Jose, CA 
into multiple street-level horizontal 
views. These authors constructed 3D 
planar polygon models for the buildings 
and terrain from the source photo- 
graphs using photogrammetry tech- 
niques rather than from blueprints or 
dimensional drawings. This was per- 
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haps the first demonstration where the 
object-texture transformation was a 
perspective projection, expressed in the 
form of the preceding collinearity equa- 
tion, rather than an affine or ortho- 
graphic projection and where it was also 
concatenated with the perspective ob- 
ject-screen transformation to form a 
single composite texture mapping trans- 
formation. These authors showed that 
this composite transformation was also 
characterized by Equation (1), Using 
this composite texture mapping trans- 
formation along with the photogram- 
metric calibration of the images, these 
authors were able to automatically map 
the appropriate areas of the images 
onto each model face without manual 
intervention to cut-out and rectify them 
first to face-on views. 

In 1981, they also presented perspec- 
tive transformations of both LANDSAT 
images and digitized high-altitude air- 
craft photographs of the Grand Canyon 
area in Arizona [Devich and Weinhaus 
1981b]. Like Tanaka, they used a regu- 
lar grid of elevation values for the 3D 
model of the terrain; however, this grid 
was purchased as a standard product 
from the United States Geological Sur- 
vey. In addition to showing transformed 
images in perspective, panoramic, and 
orthographic formats, they also pre- 
sented a pair of transformed images 
that could be viewed stereoscopically in 
3D, 

Another noteworthy early develop- 
ment was that by Dungan [1979]. How- 
ever, he transformed a computer-syn- 
thesized image of a rural setting. The 
pseudophotographic texture information 
for this synthesized image was gener- 
ated from a regular grid of elevation 
values using a combination of process- 
ing techniques that included illumi- 
nated shading based upon a diffuse 
(Lambertian) reflection model. Subse- 
quently, several other authors created 
perspective views of rural terrain set- 
tings using the diffuse shading tech- 
nique eitlier as a preprocessing step, 
like Dungan, or including it as an inte- 
gral part of the rendering process [Co- 



quillart and Gangnet 1984; JVIiller 1986; 
Robertson 1989]. 

Since the early 1980s, most examples 
have concentrated on rural settings. 
These so-called terrain rendering ap- 
proaches have been very amenable to 
the development of novel and efficient 
algorithms. Only a few examples have 
been published that have dealt with the 
more complex urban or mixed rural and 
urban setting. 

Most IPT examples have focused on 
rural settings because the terrain sur- 
face often can be treated as a single- 
valued function of elevation, that is, a 
height field. Vertical overhangs are not 
allowed or are ignored. This permits 
much freedom in the form that can be 
used to represent the 3D model of the 
terrain. The simplest form treats the 
terrain by an array of equally spaced 
elevation points or vertical posts. Some- 
times, the posts are considered to have 
a width equivalent to their spacing. 
Consequently, the terrain surface is 
represented as a uniform step function. 
Another representation that has gained 
popularity is the voxel. Voxels are es- 
sentially volume elements (cubes or 
rectangular parallelepiped columns). In 
their most general usage, they are 
linked in an octree hierarchical struc- 
ture and all sides of the voxel are impor- 
tant [Samet and Webber 1988a, b]. They 
have found perhaps their most signifi- 
cant application in volumetric rendering 
for the medical imaging field where 
they are often used to represent the 3D 
form of the human body. However, 
when they are used as a single-valued 
function to represent the surface of nat- 
ural terrain, the sides are either ig- 
nored or coded with the same texture 
value as the top. Therefore, such voxels 
also represent the terrain as a step 
function, but with horizontal step di- 
mensions that may not be uniform. In 
other words, where the terrain is rela- 
tively flat, larger voxels and wider steps 
may be used. The terrain may also be 
represented in more traditional forms 
by a mesh of planar triangles, bilinear 
rectangles, or higher-order patches. Fig- 
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a) post array model b) voxel model c) polygon 



Figure 3. Various representations of 3D models for 
the left side and a building on the right side. 



ure 3 depicts a representative mixed 
rural and urban area for each of these 
types of 3D models. 

3.2 Height Field-Based Approaches 

One can find many examples of perspec- 
tive transformations of images of rural 
settings based upon the point-array, 
post-array, or uniform column-type 
voxel representation of the terrain ele- 
vation. This is by far the most fre- 
quently used representation because it 
accommodates the simplest and most 
efficient rendering algorithms. This effi- 
ciency is achieved (1) by a preprocessing 
step to coregister and resample the im- 
age and the terrain elevation array to a 
common (map projection) coordinate 
system and scale and (2) by keeping 
both the image and terrain array in 
memory when the data sets are small 
enough. In addition, both this approach 
and the voxel-based approach discussed 
in the following are amenable to mas- 
sive parallelization. 

The registration and resampling oper- 
ations account for the texture-to-object 
transformation. This is particularly ad- 
vantageous when generating multiple 



model 

a mixed rural and urban region with the terrain on 



views for motion sequences, since such 
computations need not be repeated for 
each frame. However, there is also a 

disadvantage associated with registra- 
tion. When high-altitude satellite data 
such as LANDSAT or SPOT are used, 
there may be only a minimal affnie 
transformation required to scale, rotate, 
skew, and translate the image. How- 
ever, if multiple oblique images are 
used, then more complicated techniques 
such as orthorectification must be per- 
formed to get these images into a com- 
mon vertical format and to correct them 
for terrain elevation relief distortion. 
Furthermore, these images must be mo- 
saicked together. This may be quite 
time consuming, since it is not uncom- 
mon for the mosaicked image to be tens 
of thousands of pixels on a side. Another 
drawback is that the terrain elevation 
array often must be enlarged to match 
the finer scale of the mosaicked imag- 
ery, thereby causing an increased bur- 
den on memory and/or disk storage. 

Two categories of approaches can be 
found in the literature for the height- 
field terrain representation. The first is 
generically called ray-tracing, but is 
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Figure 4. Ray-tracing methodology. 



usually only ray-casting, because sec- 
ondary rays for reflection and/or refrac- 
tion are not generally spawned. This is 
an inverse (screen-to-object) transfor- 
mation approach. The second is what 
might be called point projection, which 
is a forward (object-to-screen) transfor- 
mation approach. These two approaches 
are depicted in Figure 3. 

3.2.1 Ray-Tracing Approaches for 
Height Fields. With the ray-tracing 
approach, every pixel in the screen (out- 
put view) is processed in^equence. This 
might be described as output data 
driven and it grows in proportion to the 
size of the screen image generated and 
to a lesser extent in proportion to the 
size and complexity of the 3D terrain 
model. A ray from the (output) eye point 
is cast through each screen pixel in se- 
quence and its intersection with the ter- 
rain is determined. Dungan's [1979] 
shaded terrain work, described earlier, 
used a ray-tracing technique. Butler 
and Harville-Hendrix [1988] and Butler 
[1989] at Visual Information Technolo- 



gies (VITec, now Connectware Inc.), and 
Nack et al. [1989a, b] at Image Data 
Corp. (now called Core Software Tech- 
nology) have independently presented 
ray-tracing examples of image perspec- 
tive transformations using variations on 
an extremely efficient algorithm called 
accelerated ray-tracing (ARTS) [Fuji- 
moto et al. 1986]. This technique uses a 
3D DDA concept, which steps along the 
projection of the ray on the ground 
plane in horizontal units equivalent to 
the terrain post spacing, decrements the 
elevation of the ray by a constant 
amount precomputed from the slope of 
the ray, and compares it with the coiTe- 
sponding elevation of the terrain. When 
the ray height makes the transition 
from above the terrain elevation to be- 
low the terrain elevation, the ray has 
pierced the terrain. See Figure 4. Then 
a nearest-neighbor or higher-order in- 
terpolation, such as bilinear, cubic con- 
volution, or windowed sine function, can 
be used to compute the {X, Y) ground 
coordinates of the intersection. Once 
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these coordinates are determined, the 
corresponding location in the texture 
image is addressed and the color of this 
pixel or interpolated color from neigh- 
boring pixels is transferred to the 
screen. In certain circumstances, speed- 
ups can be achieved for each subsequent 
pixel along a screen column by starting 
the search each time from the last inter- 
section rather than from the (X, Y) co- 
ordinate of the eye point. 

When higher order interpolation is 
used to locate the exact terrain intersec- 
tion, the terrain is essentially repre- 
sented by a smooth surface rather than 
by a step-function. Therefore, this inter- 
polation anti-aliases ridgelines; that is, 
it smooths out these "jaggies." Further- 
more, when higher-order interpolation 
is used to resample the input image, 
some texture anti-aliasing can be 
achieved (by virtue of the larger inter- 
polating kernel size) to mitigate the ar- 
tifacts produced by undersampling. This 
anti-aliasing reduces the amount of 
speckling in single scenes and blinking 
or shimmering in motion sequences that 
are typical of ray-traced obUque per- 
spective views. However, due to the fi- 
nite size of the interpolating kernel, the 
mitigating effects are of limited benefit. 
In fact, they fail where there is a large 
change in scale between the input and 
output images, for example, in the back- 
ground regions of a low oblique perspec- 
tive view. In such cases, an output pixel 
would have a very elongated "footprint" 
if projected into the input image, cover- 
ing many or even hundreds of pixels. 
Thus, a bilinear, cubic convolution, or 
windowed sine function kernel (nomi- 
nally, 2 X 2, 3 X 3, and 7x7 pixels on a 
side, resp.) would not adequately sam- 
ple the data within this "footprint." 

The height-field approach also has 
been applied to the urban setting where 
the elevations of the terrain posts are 
raised locally to account for the heights 
of buildings. However, the faces of 
buildings rendered in this manner often 
display a vertically striped appearance, 
since the texture values along the 
boundaries of the roofs are simply ex- 



tended down the sides of the buildings 
where no source texture is available. In 
other words, this approach simply as- 
signs the texture that corresponds to 
the top of the elevation posts to the 
sides of the (visible) posts. 

The approach used by the Jet Propul- 
sion Laboratory to make their early 
movies was also a ray-tracing method. 
It has been described by Hussey et al. 
[1986] and Mortensen et al. 1.1988]. Sub- 
sequently, Stanfill [1991] has intro- 
duced multiresolution (pyramid) imag- 
ery techniques into JPL's approach to 
reduce the sparkling artifact typical of 
aliasing. A precomputed lookup table 
based upon the range from the eye point 
to the terrain is used to select the ap- 
propriate image resolution level to use 
during rendering. Each output pixel is 
then rendered as a weighted blend from 
two different input image resolution 
levels so that no sudden spatial or tem- 
poral transitions in resolution are ap- 
parent. Figure 5 is an example image 
generated by this multiresolution ray- 
casting technique using a height-field 
representation for the terrain. 

Such pyramid techniques are much 
better at mitigating the aliasing arti- 
facts previously discussed because the 
input data are more completely repre- 
sented in any sampling for an output 
pixel. The downside is that they tend to 
overcompensate; that is, they include 
too many input data in their "footprint" 
sampling, since they are usually gener- 
ated with a square rather than a rectan- 
gular averaging filter. Often the result 
is too much blurring rather than too 
much aliasing. See Williams [1983J and 
Heckbert [1989, 1983] for more details. 

Devich and Weinhaus [1981b] also 
presented a novel variation on the ray- 
tracing concept. They recognized that, 
for a camera with a horizontally ori- 
ented optical axis, columns and rows on 
the screen correspond to azimuth and 
elevation angles. Therefore, before ray- 
tracing, they processed both the terrain 
elevation array and the image into a 
polar coordinate system centered at the 
iXy Y) position of the eye point. They 
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Figure 5. Computer-generated view of the 5 mi high volcano, Maat Mons, on Venus. Source data 
included coloi'ized radar imagery acquired by the MageHan spacecraft and altimeter elevation data in 
height-field format. Perspective view generated by JPL/NASA and photo obtained from Newell Color 
Lab. 



also processed the elevation values in 
the terrain array, first to correct for 
earth curvature and then into values 
representing screen-line coordinates 
which are proportional to the tangent of 
the elevation angles. Ray-tracing was 
thereby transformed from incrementing 
both X and Y ground coordinates to 
incrementing simply radial distance, 
that is, along a single dimension. Fur- 
thermore, no decrement in the ray ele- 
vation was required, because of the con- 
version of the terrain elevation values 
to a quantity more related to elevation 
angle. Thus, extra computations associ- 
ated with the Cartesian-to-polar and el- 
evation-to-elevation-angle processing 
were traded for more efficient ray-trac- 



ing, Weinhaus and Walterman [1990] 
have pointed out that the Cartesian-to- 
polar transformation need only be set 
up once in the form of a spatial transfor- 
mation lookup table and can be used for 
all subsequent translations of the eye 
point fbut not for changes in view direc- 
tion). For the case of tilted views, Dev- 
ich and Weinhaus implemented an ex- 
tra one-dimensional transformation 
along each row of the screen space im- 
age to correct the geometry from that of 
a horizontal view. Anti-aliasing was 
achieved by processing multiple resolu- 
tion versions of the input into range 
annuli in the output and by using high- 
er-order resampling techniques. Ridge- 
lines were specially anti-aliased by 
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blending information from foreground 
and background pixels using area cover- 
age weighting factors. 

Paglieroni and Peterson [1992] have 
presented a ray-tracing method that 
adaptively optimizes each step size dur- 
ing the ARTS algorithm. Their method 
involves preprocessing the elevation 
data using distance transforms [Borge- 
fors 1986; Paglieroni 1992]. For each 
possible height value in the digital ele- 
vation array, they extract a horizontal 
cross-sectional slice in the form of a 
binary image. This binary image codes 
regions above and regions equal to or 
below the given elevation value. The 
X-Y Euclidean distance from each 
"above" pixel to the closest "equal to or 
below" pixel in a plane is then computed 
and stored. In concept, this would form 
a set of hierarchical distance, images, 
one image for each possible height value 
within the data. During ray-tracing, the 
(X, y, Z) position at a given step could 
be used to look up the next step's hori- 
zontal increment from within the set of 
hierarchical distance images. However, 
to save on disk storage, for each pixel in 
the elevation model, they form a para- 
metric representation for the hierarchi- 
cal data. This approximation sets a 
lower bound on the step size. Thus, in 
place of the full sequence of distances as 
a function of height values, only a slope 
and intercept are stored for each pixel 
in the digital elevation array. The dis- 
tance to use for any given step is then 
computed from these slope and inter- 
cept values rather than from an actual 
lookup into the hierarchical distance 
images. For the data tested, Paglieroni 
and Peterson found speedup factors for 
their algorithm of about 40 and 6 when 
compared to a uniform step, accelerated 
ray-tracing method, and to an hierarchi- 
cal step, octree-like ray-tracing method, 
respectively. 

3,2,2 Point Projection Approaches for 
Height Fields. In the point projection 
approach, the registered image and ter- 
rain elevation array are divided into 
corresponding profiles and these dual 



profiles are processed one pair at a time 
as follows. First, each pixel in the ter- 
rain elevation profile is projected from 
object space to screen space where it is 
usually truncated or rounded to the 
closest integer coordinate. Then the tex- 
ture value from the corresponding pixel 
in the image profile is assigned to the 
specified screen coordinate. This process 
is input data driven and grows in pro- 
portion to the size of the registered 3D 
terrain model and source (texture) im- 
age. 

One generally finds two common 
forms of hidden pixel removal used in 
conjunction with the point projection 
approach: the painter's algorithm and 
the floating horizon algorithm. How- 
ever, in either case, the profiles typi- 
cally are linear slices through the regis- 
tered digital elevation array and texture 
image oriented orthogonal to the projec- 
tion of the output view's optical axis 
onto these data sets. Alternately, they 
may simply be rows or columns of this 
dual data, if properly sequenced. Figure 
6 depicts both kinds of point projection 
algorithms as discussed in the follow- 
ing. 

In the painter's algorithm, the pro- 
files are processed starting with the 
most distant profile from the eye point 
and advance towards the profile closest 
to the eye point. When textures corre- 
sponding to different terrain elevation 
profiles contend for the same screen 
pixel, the current one (associated with 
the closest profile to the eye point) al- 
ways overwrites the previous one. 

In the floating horizon technique, the 
profiles are processed in the opposite 
order; that is, they advance from the 
profile closest to the eye point towards 
the profile furthest from the eye point. 
A floating horizon buffer is maintained 
as a mechanism to eliminate hidden 
pixels. As each profile is processed, this 
buffer is potentially updated and saves 
the screen line coordinate associated 
with the current horizon for each screen 
sample coordinate. Any projected point 
that falls below the current horizon is 
ignored. The current horizon is only up- 
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(d) 



Figure 6. Point projection approaches, (a) Terrain image: brightness encoded elevation. Higher 
elevations are darker; (b) perspective view (every 8th row): profiles processed back to front for Painter's 
algorithm; profiles processed front to back for Floating Horizon algorithm; (c) Painter's algorithm: 
partially processed; (d) Floating Horizon algorithm: partially processed showing current location of 
floating horizon. 



dated whenever a projected point falls 
above the current horizon. When tex- 
ture pixels corresponding to different 
terrain elevation profiles contend for 
the same screen pixel on this horizon, 
the current one is usually ignored in 
favor of the previous one (associated 
with the closer profile to the eye point). 
Alternately, an area weighted averaging 



(A-buffer) technique may be used for 
each horizon screen pixel to store and 
blend values from texture pixels associ- 
ated with more than one profile. 

A disadvantage common to the point 
projection technique occurs when trans- 
forming terrain elevation and texture 
arrays for close-range viewing. In this 
case, without special techniques such as 
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described by Fant [1986], undersam- 
pling in screen space will leave unfilled 
pixels. For nearly horizontal and low- 
altitude viewing conditions, this often 
occurs in the foreground part of the 
output image where the source data are 
magnified. The opposite situation or 
oversampling in screen space is a com- 
mon occurrence in the background part 
of the output image or for overall dis- 
tant viewing where source data are 
minified. In this case, many texture pix- 
els are transformed to the same screen 
location. This is definitely inefficient 
unless multiresolution source imagery 
techniques are used. One advantage 
typical of the point projection technique 
is that ridgeline anti-aliasing is easily 
achieved with the floating horizon ap- 
proach when it uses an A-buffer. 

Tanaka's [1979] example described 
earlier used the painter's method. 
Junkin [19821 at NASA also has pre- 
sented examples of point projections of 
LANDSAT images in perspective, but 
he used the floating horizon method. 
Other examples of the point projection 
method have been presented by Fish- 
man and Schachter [1980] at General 
Electric and Smedley et al. [1989] at 
Lockheed. However, their examples 
transformed synthetic images generated 
by a terrain elevation diffuse shading 
algorithm. Smedley's examples used 
bathymetric elevation data. Bernstein 
et al. [1984] and Pareschi and Bernstein 
[1989] at IBM presented transforma- 
tions of images of the San Francisco Bay 
area and of Mt. Etna, respectively, but 
projected them, orthographically rather 
than perspectively. Carlotto and Hartt 
[1989] at TASC presented orthographic 
projections of images of Mars using a 
variation on the floating horizon tech- 
nique that they implemented in the 
form of a preprocessing step to identify 
hidden pixels. They obtained their ele- 
vation data using a shape-from-shading 
technique. 

A novel orthographic point projection 
approach also based upon horizon infor- 
mation has been developed by Dubayah 
and Dozier [1986] at the University of 



California Santa Barbara. In this tech- 
nique, the terrain elevation grid is pre- 
processed to compute an array whose 
intensity at each element represents the 
angle to the local horizon. During point 
projection, the elevation angle for a 
given terrain point is tested against this 
preprocessed horizon angle to determine 
whether it is visible. These authors also 
developed an accumulation technique to 
address the problem of unfilled screen 
pixels. Each registered terrain elevation 
and texture pixel is projected to the 
screen, but the coordinates are kept to 
floating point precision rather than sim- 
ply truncated or rounded to the inte- 
gers. Then a fraction of the color is 
assigned to each of the four closest 
screen pixels according to an inverse 
distance weighting scheme. Each screen 
pixel can accumulate fractional color 
from more than one texture pixel. Fi- 
nally, the accumulated color at a screen 
pixel is normalized by the sum of the 
inverse distances accumulated for all 
the texture pixels that contributed to it. 

Another point projection approach 
that mitigates hole-like artifacts has 
been presented by Petersen et al. [1990] 
at Brigham Young University. In their 
back-to-front approach, the complete el- 
evation post is rendered to the screen by 
projecting both its top and bottom and 
subsequently connecting a line between 
the two points. Then each screen pixel 
along the line is assigned the color of 
the source texture pixel that corre- 
sponds to the elevation post. 

A novel variation similar in concept to 
the ID approach of Devich and Wein- 
haus has been developed by Robertson 
[1987] at CSIRO, but is based upon the 
point projection approach. Robertson 
used a combination of rotational and 
rhombic deformations of the texture im- 
age and terrain elevation array, rather 
than the Cartesian-to-polar one, to 
transform to a domain where the projec- 
tion became one-dimensional (i.e., along 
rows or columns in the transformed do- 
main). However, he used a back-to-front 
projection technique rather than a ray- 
tracing method to render the output. 
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3.3 OctreeA^oxe^Based Approaches 

Perspective transformations of images 
using octree encoded voxels also have 
been used. One finds both forward pro- 
jections from the voxels to the screen 
and inverse projections from the screen 
to the voxels. The forward transforma- 
tion technique is similar to the point 
projection technique previously de- 
scribed, except all nodes of the octree 
are projected at their appropriate levels 
of detail. Indexing and projection of the 
nodes of the octree may progress either 
from the back towards the front or from 
the front towards the back. The inverse 
transformation technique uses the ray- 
tracing concept, but must intersect and 
identify the appropriate node of the oc- 
tree. 

Not only does the voxel carry the 3D 
structure, but it also can be coded with 
source image color or other attributes 
about the model that would be useful 
for simulating views by such nonvisual 
sensors as infrared and radar. Further- 
more, when multiresolution source im- 
agery is used, a color can be assigned to 
each node of the structure using the 
pixel color at the corresponding image 
resolution level. The octree hierarchy 
allows the voxel approach to deal with 
vertical faces of urban 3D objects in 
addition to the surface of the terrain. 
However, unless very fine voxels are 
used, this technique will not represent 
sloped surfaces of urban models very 
smoothly. 

Nack et al. [1989a, b] used a software- 
based ray-tracing approach and have 
presented perspective transformations 
showing the sides of individual build- 
ings at high resolution as well as those 
of rural settings. Figure 7 is an example 
image generated by this type of ray- 
tracing approach using a column voxel 
representation for the terrain. On the 
other hand, Scuderi and Strome [1988] 
have presented voxel projection trans- 
formations of both rural and urban set- 
tings. These output images were gener- 
ated on prototype hardware developed 
by Hughes Aircraft Co. However, their 



results for the urban environment also 
displayed the same striping artifacts on 
the sides of the buildings as was de- 
scribed earlier, due to the use of simple 
column voxels and vertical photography, 

3.4 Polygon/Patch-Based Approaches 

The traditional polygon and patch rep- 
resentation of 3D models has also been 
used to perform perspective transforma- 
tions of images. In this case, the follow- 
ing techniques similar to those used for 
ordinary texture mapping can be ap- 
plied: back-to-front or front-to-back 
depth-priority, depth-buffer, scan-line, 
polygon subdivision, and ray-tracing. 

An example of an early polygon-based 
approach was Sun Microsystem's 
MAPVU demo 11989] which ran on their 
TAAC-1 accelerator. It processed a reg- 
ular 3D mesh of triangles in a back-to- 
front (painter's) fashion. Each vertex of 
a triangle was assigned the color of the 
corresponding pixel in the image tex- 
ture. Triangle vertices were projected to 
the screen and the color for each screen 
pixel within a projected polygon was 
interpolated using the traditional color 
method. A quality parameter controlled 
subsampling of the data. It permitted 
coarse 3D mesh and texture data to be 
used thereby increasing the rendering 
rates for rapid previews. However, this 
occurred at the expense of poorer tex- 
tural detail, since no other source image 
data were used to fill the triangles. 

Subsequent polygon-based ap- 
proaches such as those of SGI and Star- 
dent (now Kubota Graphics) could fill 
the interior of each triangle in a coarse 
mesh with high resolution texture from 
the source image. The approach taken 
in this case properly interpolated the 
texture coordinates for each projected 
triangular polygon from texture coordi- 
nates stored at the vertices. Then the 
interpolated texture coordinates were 
used to look up the color in the source 
image. A depth-buffer was used to re- 
move hidden pixels. 

In many high resolution, nonreal-time 
polygon-based terrain rendering appli- 
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Figure 7. Perspective view of the area, around Pasadena, CA, computer-generated from coregistered 
30 m LANDSAT Thematic Mapper imagery and voxel format terrain elevation data. Courtesy of Image 
Data Corp. (now Core Software Technology ©). 



cations, the grid of elevation values is 
preprocessed to i^esample it to the same 
resolution as the (rectified) image. This 
is advantageous in the following ways. 
First, it means that texture coordinate 
interpolation is unnecessary, since 
there are no interior texture pixels to 
access. It suffices in this case simply to 
assign the colors of the image pixels to 
their corresponding mesh vertices and 
interpolate them across the polygon using 
the traditional color method. Second, if 
higher-order interpolation techniques 
are used in the terrain resampling pro- 
cess, then the terrain will display a 
smoother appearance, although not 
truly any more accurate than in its orig- 
inal representation. Other advantages 
associated with resampling the texture 



data and 3D model geometry to a com- 
mon coordinate system have been dis- 
cussed by Cook et al. [1987]. The main 
disadvantage is that the complexity 
(i.e., number of polygons in the terrain 
3D model) rises dramatically. Thus, 
rendering can become extremely time 
consuming for large, high resolution 
data sets. Techniques that generate and 
use multiresolution terrain elevation 
data are therefore necessary to mitigate 
this problem. 

Moreover, when the data sets are too 
large to be totally contained in memor}^, 
tiling (i.e., partitioning the data into 
blocks) also becomes important. For ex- 
ample, Blinn [1990] has explained tiling 
and paging strategies that he has used 
to perform efficient texture mapping of 
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astronomical photographs of the planets 
onto spherical models. 

Gelberg et al. [1988] and Hughes 
[1991] each used combined tiled and 
multiresolution (pyramid) approaches to 
generate TASC's Calgary video and Ap- 
ple's Mars Navigator videos, respec- 
tively. Each started with a terrain ele- 
vation grid that was registered to a very 
large rectified image (36 milHon pixels 
in Gelberg's example and 145 million 
pixels in Hughes' example). Gelberg 
used a simple tiled pyramid approach 
and Hughes used the MIP map pyramid 
approach. 

In a simple tiled pyramid approach 
[Tanimoto and Pavlidis 1975; Burt 
1981], multiple versions of the source 
terrain elevation grid and/or imagery 
are created usually with a factor of two 
steps in resolution, stopping at a speci- 
fied block size. Each version is then 
subdivided into blocks, reformatted in 
scan-line order and stored as a hierar- 
chy of tiles, each containing one block of 
data. In the MIP map approach [Wil- 
liams 1983], the source terrain eleva- 
tion grid and/or imagery are first di- 
vided into blocks, A pyramid is then 
formed for each block, typically stopping 
when the size is 2 X 2 pixels. Finally all 
resolution versions of a given block are 
reformatted in scan-line order and 
stored successively in the same tile. In 
the simple tiled pyramid, the number of 
resolution levels is given by (1 + log2 
(original image size/block size)). In the 
MIP map pyramid, the number of reso- 
lution levels is given by (1 + log2 (block 
size/2)). 

An advantage of the MIP map ap- 
proach over the simple tiled pyramid 
approach is that all resolutions of the 
data are always readily available for 
any MIP map tile in memory. This is 
especially convenient when it is desired 
to blend data from two successive reso- 
lutions so that sudden transitions in 
resolution are not detected. When used 
in conjunction with bilinear resampling, 
this technique is commonly referred to 
as trilinear interpolation. On the other 
hand, a disadvantage is that memory is 



less efficiently allocated with the MIP 
map approach. This occurs because the 
higher resolution data also must be car- 
ried along for those MlP-mapped tiles 
corresponding to the background areas 
of the output scenes where only low 
resolution data are needed. When mem- 
ory is limited, more tile paging is re- 
quired due to the smaller amount of low 
resolution data available in any one tile. 
A compromise approach might be to 
store a sequence of MIP maps for each 
block of the image where each MIP map 
contains only two successive resolu- 
tions. However, this would increase the 
total data load from a factor of 4/3 times 
the original image and/or terrain eleva- 
tion array size to 5/3 times the original 
image and/or terrain elevation array 
size. 

In Hughes' approach, only those ter- 
rain elevation tiles that were found to 
lie at least partially within the output 
image's ground "footprint" (projection 
onto the X, Y ground plane) were ren- 
dered. The terrain elevation model and 
imagery resolutions were selected as a 
function of distance between the terrain 
elevation tile and the eye point so that 
coarse resolution could be used for more 
distant regions. Terrain elevation tiles 
were then tessellated into triangular 
meshes at the specified resolution. Then 
color values from the corresponding res- 
olution imagery pixels were assigned to 
each triangle vertex. An SGI computer 
was used to perform the rendering by 
simply interpolating the color values 
across the triangles and by using a 
depth-buffer to remove hidden pixels. 

In Gelberg's approach, terrain eleva- 
tion tiles were processed by ordering 
them according to distance in a back-to- 
front manner after eliminating those 
tiles that were not at least partially 
within the viewing frustum of the out- 
put image. The appropriate resolution 
data were determined from the ratio of 
the number of quadrilateral polygons to 
the number of screen pixels. A Pixar 
computer was used to perform the ren- 
dering using the ChapReyes software 
[Cook et al. 1987]. This software subdi- 
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vides each 3D model primitive into 
quadrilateral-shaped planar micropoly- 
gons and renders them using a depth- 
buffer to remove hidden pixels. 

Kaneda et al. [1989] in Japan also 
generated perspective transformations 
of rural settings using a quadrilateral 
mesh to represent the 3D model of the 
terrain. Their algorithm used a back-to- 
front depth-priority technique for ren- 
dering. However, they used a specially 
preprocessed, radially organized quadri- 
lateral mesh rather than one in a recti- 
linear format. This was done to gener- 
ate uniformly sized quadrilaterals in 
screen space. Fewer quadrilaterals are 
needed this way, since they are corre- 
spondingly larger in object space as 
they progress farther into the distance. 
These authors also created multiple res- 
olution versions of the vertically ori- 
ented (aerial) image and selected the 
appropriate resolution to use for each 
quadrilateral so that high resolution 
texture was not wasted for the back- 
ground regions of the scene. 

Another quadrilateral-based approach 
has been presented by Whiteside et al. 
[1987] and Whiteside [1989] at DBA, 
but it used a combination of depth- 
buffer and accumulation-buffer tech- 
niques for rendering. In this implemen- 
tation, more than one source image was 
used to render the area of interest, but 
they were first orthorectified and mosa- 
icked to a vertical format. Multiple res- 
olution versions of the mosaicked image 
were created subsequently and used as 
required to avoid oversampling of the 
input texture. This was achieved by pro- 
jecting each 2X2 set of terrain eleva- 
tion posts into input (texture) space and 
screen space and selecting the texture 
resolution that nearly matched the 
quadrilateral areas in the two domains. 
Then a forward projecting, bilinear 
transformation rather than a true per- 
spective transformation was used to 
map the rectangular-bounded input tex- 
ture areas to quadrilateral-bounded 
output screen areas. Screen-space anti- 
aliasing was nicely achieved for the 
oversampling case using an accumula- 



tion buffer. (No discussion of the under- 
sampling case was presented. However, 
the resampling scheme described by 
Fant [1986] should be applicable to the 
approach advocated by these authors 
and would mitigate this problem.) 
These authors also were able to render 
3D cultural features with rectangular 
faces, such as buildings. The textures 
for these faces, however, had to be 
cropped from the images and geometri- 
cally preprocessed into face-on views be- 
fore they could be used. 

In general, quadrilaterals are espe- 
cially difficult 3D primitives to use, be- 
cause they are not necessarily planar. 
Even when their edges are constrained 
so that they form bilinear surfaces, they 
may still present ridgelines for a partic- 
ular view. Thus, back-facing portions of 
the surfaces must be taken into account 
if they are to be handled rigorously. 
Generally, triangles (and planar quadri- 
laterals) are much easier to deal with, 
because testing for backfacing condi- 
tions need only be done once per surface 
rather than at every interior pixel. It 
has therefore been necessary when 
dealing with nonplanar quadrilaterals 
to use approximations and other special 
techniques, such as polygon subdivision, 
in order to avoid these difficulties. 



3.5 Approach Comparison 

Table 3 presents a comparison of the 
various categories of approaches that 
have been discussed. The approaches 
have been categorized as mentioned 
earlier by the geometry primitives and 
rendering methods used. The compari- 
son identifies the major advantages and 
disadvantages of each category. Too 
many algorithmic variations within cat- 
egories have been presented to go into 
details for each. The advantages and 
disadvantages that have been presented 
here focus on issues such as resulting 
quality (e.g., aliasing), processing effi- 
ciency, and the ability to handle urban 
(man-made) as well as rural (natural 
terrain) features. 



ACM Computing Surveys, Vol. 29, No. 4, December 1.997 



Texture Mapping 3D Models • 351 



Table 3. Algorithmic Comparison 



Geometry 


Rendering 


Advantages 


Disadvantages 


Height field 


Painter's 


• Algorithmic simplicity 


• Minimal antialiasing 

• No handling of (urban) 
vertical faces 


Height field 


Floating horizon 


• Good ridgeline antialiasing 


• Minimal other 
antialiasing 

• No handling of (urban) 
vertical faces 


Height field 


Ray trace 


• Efficient handling of large 
terrain sets 

• Amenable to speed 
optimizations 

• Amenable to parallelization 


• Minimal to moderate 
antialiasing 

• Poor handling of (urban) 
vertical faces 


Voxel 


Ray trace 


• Can handle (urban) vertical 
faces 

• Amenable to speed 
optimizations 

• Amenable to parallelization 


• Geometry data base 
increases substantially to 
handle (urban) objects 


Polygon 


Depth buffer 


• Good antialiasing using 
mul tires data 

• Handles urban & terrain 
objects equally 


• Limited parallelization 



4. IPT SYSTEMS 

4.1 Nonreal-Time Systems 

The image perspective transformation 
capability is relevant to a number of 
disciplines: urban planning, architec- 
ture, law enforcement, industrial visu- 
alization, and military training to name 
just a few. When used, for example, in 
mission planning and rehearsal sys- 
tems, IPT techniques must accurately 
portray the real-world scene. Any dis- 
tortions in the transformed images, 
such as inaccurately placed doorways on 
building faces or misplaced roads on the 
terrain, cannot be tolerated. Further- 
more, these systems must be easy to use 
and require as little manual interven- 
tion as possible. For mission planning 
systems, real-time scene generation is 
desirable, but not mandatory. Conse- 
quently, increased output scene genera- 
tion times may be traded off against 
higher output scene quality and accu- 
racy. 

Although much research has been 
conducted towards fully automated ex- 
traction of 3D models of cultural fea- 



tures this task still remains primarily 
a manual one. However, it can be made 
user-friendly in several ways. The first 
way is to use a modern man-machine 
interface that permits the modeler to 
construct 3D models interactively and 
simultaneously displays them as over- 
lays on the source images. The second 
way is to use computer-assisted or semi- 
automated techniques to extract 3D 
models from the imagery IMueller and 
Olson 1993]. The third way is to elimi- 
nate potentially unnecessary prepro- 
cessing operations, especially those that 
can be folded into the basic projective 
geometry transformations used during 
the rendering. These include: orthorec- 
tification and mosaicking of ground cov- 
erage images, resampling of the terrain 
elevation data to match the imagery 
resolution, and rectification of images of 
the sides of urban features into face-on 
views. Such rectifications introduce po- 
tentially unnecessary resampling of the 
imagery that, in addition, degrade the 



^ See, for example, Herman and Kanade (19861, 
Irvin and McKeown [1989], Walker and Herman 
[1988], and DARASISTO [1992, 1993]. 
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results. Several nonreal-time, worksta- 
tion-based systems have made advances 

in these areas. These include systems 
developed by TRW/ESL, General Dy- 
namics Electronics Corp., SRI, and the 
University of California at Berkeley. 

TRW's system renders the output 
scenes by computing texture coordi- 
nates for every screen pixel using an 
incremental scan-line method to evalu- 
ate the fractional linear perspective 
transformation of Equation (1). This 
transformation is able to account for the 
oblique geometry of the source texture 
images as well as the viewing perspec- 
tive associated with the output. The ar- 
ticles by Devich and Weinhaus [1980, 
1.981a] and Weinhaus and Devich [1997] 
showed how to derive the fractional lin- 
ear transformation coefficients for a pla- 
nar polygon independent of the number 
of its vertices without the need to store 
u, V {or w) texture coordinates for each 
polygon. The only information needed is 
the camera parameter model for the 
texture image and the equation of the 
plane for the polygon. In fact, the four 
coefficients defining the plane need not 
be carried with the polygon, since they 
can be generated from the world coordi- 
nates of the vertices. 

In this system, none of the imagery, 
either for the texture for the ground 
covering or for the sides of urban fea- 
tures, has to be rectified or mosaicked. 
A simple tiled pyramid is created for 
each source image and the appropriate 
tile (or tiles) is automatically selected to 
cover each surface at an appropriate 
resolution for anti-aliasing. A deferred 
texturing, depth-buffer technique is 
used and various resampling techniques 
may be selected to trade quality versus 
speed, including supersampling and/or 
EWA on a pyramid. The EWA technique 
[Greene and Heckbert 1986] is a very 
good method for anti-aliasing, since it 
samples the input data within an ellip- 
tical "footprint" region that represents 
the projection of a given (circular) out- 
put pixel. The size and orientation of 
the ellipse adapts to the geometry and 
depends upon the orientation and loca- 



tion of both the input and output cam- 
eras and the orientation of the 3D 
model surface upon which it is pro- 
jected. 

The terrain elevation data, which 
originate from a regular grid of eleva- 
tion values, are automatically reformat- 
ted into a multiresolution triangular 
surface mesh for each output frame. 
This allows coarser triangles to be used 
in the background of output images and 
finer ones to be used in the foreground. 
However, before rendering any frame, 
surfaces from two levels are blended. 
This prevents sudden transitions in 
level of detail from being observed in 
motion sequences. Urban features are 
modeled by interactively adjusting 
"elastic" wireframe models on one or 
more source images simultaneously. 
Figures 8 and 9 show examples of an 
image perspective transformation for an 
urban and rural area, respectively pro- 
duced by this system using the polygon 
representation for the terrain elevation 
model and the cultural objects. 

The SOCET SET system, built by 
General Dynamics Electronics Corp. 
[Inman and IMcMillan 1990] uses stereo 
image pairs as the source of its 3D infor- 
mation. A combination of automatic ste- 
reo compilation followed by manual ed- 
iting can be used to generate the terrain 
elevation model. The construction of 3D 
cultural models is performed interac- 
tively on this system using a stereo- 
scopic monitor for visualization or 
by using semi-automated techniques 
[Mueller and Olson 1993]. For example, 
the vertices of the top of a rectangular 
parallelepided-shaped building can be 
identified manually with a "floating" 
cursor. Then a wire-frame graphic is 
draped automatically about the bound- 
ary of the building with the bottom por- 
tion "buried" below the ground. 

The Cartographic Modeling Environ- 
ment [Hanson et al. 1987; Hanson and 
Quam 1988], developed by Hanson and 
Quam at SRI, was created to support 
research on interactive, semi-automatic, 
and automatic computer-based carto- 
graphic activities. This system uses 
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Figure 8. Computer-generated perspective transformation of downtown San Jose, CA. Perspective 
view generated by ESL (now TRW Sunnyvale, CA). (a) high oblique aerial source image (1 ft resolution); 
(b) subsection of the source image with 3D models overlaid; (c) low oblique w^re-frame perspective view 
of 3D models; (d) low oblique image perspective transformation output. © TRW, Inc. 



MlP-mapped imagery to cover the ter- 
rain and urban models. Gridded terrain 
elevation data are converted into a mul- 
tiresolution mesh of triangular surfaces; 
however, before rendering any frame in 
a motion sequence, elevation data from 
two levels are blended to present a 
smooth transition. Terrain rendering is 



achieved using a floating horizon 
method similar to that described in 
Anderson [1982] or Wang and Staudham- 
mer [1990] in combination with an A- 
buffer. A depth-buffer also is main- 
tained so that cultural models may be 
rendered subsequently. 

This system has a very extensive 
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Figure 9. High and low oblique, computer-generated perspective views of the area around Irish Canyon, 
CO. Source data were composed of 20 m SPOT imagery and polygonalized 10 m terrain elevation data that 
were vertically exaggerated by a factor of 5. Source data courtesy of STX Corp. Perspective views 
generated by ESL (now TRW Sunnyvale, CA) .© TRW, Inc. 
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menu of interactive modeling and ma- 
nipulation tools. For example, wire- 
frame models may be rotated about 
world axes, about any object edge, or 
about a line from an object vertex to the 
sun*s position. It may even be moved 
along the ray from the eye point to the 
object. Shadow mensuration is an op- 
tion that can be used to help in 3D 
model construction. Also, shadows can 
be rendered into a graphic overlay on 
the transformed view. Semi-automated 
2D feature extraction tools, such as road 
following and the detection of bound- 
aries of buildings whereby the user 
starts the extraction process with a 
manual cue,' also are included. SRI also 
has experimented with automatic ex- 
traction of 3D models of buildings and 
used their system to portray the results 
[Fua and Hanson 1989]. 

The University of Cahfornia at Berke- 
ley's Facade System developed by De- 
bevec et al. [1996] was designed to 
model and render architectural scenes 
from multiple photogi^aphic inputs. Its 
objective is to create high quality out- 
puts of a small number of detailed ar- 
chitectural structures rather than large 
areas of rural and/or urban content. In 
this system, 3D models composed of 
parametric primitives such as blocks, 
pyramids, and the like are manually 
composited to approximate the geome- 
try of the architectural structure using 
the imagery as a visual guide. The focus 
is on linking the right shapes together 
without putting significant effort into 
setting the exact size and orientation. 
Then a few corresponding edges are lo- 
cated in the source imagery and on the 
3D models. A nonlinear least square fit 
is then performed to recover simulta- 
neously the parameters (size, orienta- 
tion, etc.) of the primitives in the 3D 
architectural model as well as the cam- 
era location and orientation parameters 
for each of the source images. 

A novel view-dependent texture-map- 
ping technique is used to render the 
architectural model. Multiple photo- 
graphs are projected onto the model in 
order to texture its surface completely. 



(An image-space shadow-mapping algo- 
rithm based on a 2-buffer is used to 
keep track of obscurations.) However, 
since the photographs overlap, the ren- 
dering algorithm must decide which 
photograph or photographs to use at 
each output pixel. Here, a weighted av- 
erage of the textures from the overlap- 
ping images is used, where the weights 
are the angular deviations of the view- 
ing vectors of each source image from 
that of the output view. Moreover, to 
avoid visible "seams," the weights are 
ramped near the boundaries of source 
photographs. Optionally, a model-based 
stereo correlation algorithm may be 
used to refine the 3D model to include 
finer detail such as recessed windows, 
and the like, and to increase the fidelity 
of the texture-mapped rendering. To 
achieve this result, the base 3D model is 
first used to create an output view sim- 
ilar to the orientation of one of the 
existing source images. These two im- 
ages are then used as input to the auto- 
mated stereo correlation algorithm. The 
stereo correlation produces a disparity 
image that can be converted into a 
depth map which in turn is used as a 
refined 2-buffer for re-rendering the 
output view. 

4.2 Real-Time Systems 

In contrast, the training industries and 
in particular the commercial and mili- 
tary flight and ground vehicle simula- 
tion industries have traditionally re- 
quired real-time scene generation rates 
in their visual systems and, historically, 
have traded off quality and realism to 
achieve these rates. These visual sys- 
tems produce interactively controlled 
moving images that depict on the simu- 
lator display what the pilot or driver 
might see if he or she were looking 
"out-the-window" or at a monitor for a 
sensor mounted on the vehicle. In the 
interest of speed, both data and algo- 
rithms are trimmed to the bare-bones 
level for use by such systems. The de- 
velopment of this technology histori- 
cally took two different paths, direct 
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and hybrid. The direct approaches were 
simply real-time implementations of 
the previously described polygon- 
based computer graphics techniques. 
The hybrid approaches merged com- 
puter graphics techniques with pre- 
stored digital images. However, both 
approaches ultimately converged on 
the use of certain fundamental proce- 
dures, notably the use of digital or- 
thophoto mosaics and registered digi- 
tal terrain elevation data. 

In the early 1980s, researchers at 
Honeywell [Scott 1983; Erickson et al. 
1984; Baldwin et al. 1983] developed 
one hybrid approach that they called 
the Computer Generated Synthesized 
Imagery (CGSI) technique. This ap- 
proach combined the best qualities at 
the time of the Computer Graphics Im- 
agery (CGI) technique and the Com- 
puter Synthesized Imagery (CSI) tech- 
nique. The CGI technique, namely, 
coloring and illumination-based shading 
of polygon models permitted full free- 
dom of motion through the gaming area 
or database. However, it lacked realism. 
The CSI technique used high resolution 
digitized photographs as backgrounds, 
for example, for artillery training with 
computer-generated target models mov- 
ing around the scene. It attained a high 
degree of realism, but basically was lim- 
ited to a single viewpoint, although 
some motion could be simulated by pan- 
ning and zooming the background im- 
age. The CGSI technique combined the 
full freedom of motion of the CGI tech- 
nique with the realism of the CSI tech- 
nique. This was achieved by using the 
computer graphics-generated imagery 
as the framework and inserting digi- 
tized photographs of real-world objects, 
such as buildings, trees, and targets at 
the appropriate locations in the data- 
base. Multiple views of these objects 
were photographed, digitized, and pre- 
stored on optical video disks. During 
real-time image generation, when a par- 
ticular object was needed, the most ap- 
propriate view of it was retrieved and 
warped to proper perspective according 
to Equation (1) for the specific output 



image. Then it would be inserted into 
the previously computed CGI back- 
ground with proper occlusions and bor- 
der blending. In effect, this combination 
of image warping and insertion is equiv- 
alent to texture mapping the image onto 
a planar 3D rectangle that has been 
placed in the database at the desired 
location and orientation and subse- 
quently viewing it from the desired eye 
point. 

Meanwhile, researchers at LTV Mis- 
siles and Electronics [Hooks and De- 
varajan 1981a] (now Lockheed-Martin- 
Vought Systems (LMVS)) developed 
another type of hybrid approach that 
they called the "pseudo-film" approach. 
This monochrome image technique in- 
terpolated new output views at real- 
time rates from a small set of specially 
collected, oblique perspective photo- 
graphs. These photographs were ac- 
quired at locations corresponding to a 
3D matrix of eye points throughout 
some area of interest in the real world. 
Before real-time interpolation by the 
image generator part of the system, 
these oblique perspective photographs 
were converted into large (typically 
thousands of pixels on a side) electronic 
images using a high resolution, digital 
scanning process and then were stored 
on analog video disks. When the real- 
time simulation was started, the system 
would know at any given time what the 
aircraft position and attitude were. It 
would then retrieve the most appropri- 
ate oblique perspective image stored in 
the video disk bank and then warp it 
according to Equation (1) to simulate 
the pilot's or sensor's view of the gam- 
ing area at that instance in time. Since 
these operations were accomplished in 
real-time, it would give the illusion of 
smooth interactive motion through the 
gaming area. An improved version of 
this system using color imagery and dis- 
plays as well as multiple output chan- 
nels has been described by Hooks and 
Devarajan [1981b] and by Mudunuri 
and Hooks [1985]. 

The warping operation used to inter- 
polate between successive oblique 
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source images in the LTV simulations 
(on their TOPSCENE Model 3500) con- 
sisted primarily of zooming, panning, 
and 2.5D perspective changes. The 
zooming operation approximated for- 
ward motion, the panning operation ap- 
proximated lateral motion, and the 2.5D 
perspective changes allowed smooth 
transitions between successive input ob- 
lique images. Since the oblique input 
images were acquired in proper perspec- 
tive, they were constructed to have cor- 
rect elevation information built in for 
both the terrain and urban features. 
Therefore, the process of warping the 
source image according to Equation (1) 
to achieve limited changes in the per- 
spective of the output views was as- 
sumed to result in a good approximation 
of the true perspective view for the 
given output eye point. As the simula- 
tion proceeded through the gaming 
area, the system would make the transi- 
tion at the appropriate time from using 
one oblique source image to the next one 
in sequence. The warping process used 
here is again equivalent to texture map- 
ping the source image onto a single pla- 
nar 3D surface, in this case, the oblique 
imaging plane of the source camera, but 
seen from a slightly different eye point 
than for that of the original image. 

Eventually, the researchers at LTV 
[Devarajan and Chen 1986; Devarajan 
1989] generated the large oblique "pseudo- 
film" perspective images from mosaics 
of overhead photographs rather than by 
collecting actual oblique photography. 
Likewise, the researchers at Honeywell 
[Hopkins and Cooper 1988; Mudunuri 
1989] (now Hughes Training Inc.) even- 
tually replaced the background imagery 
generated by the CGI technique with a 
sequence of computer-generated oblique 
Niewplane" images similar in nature to 
LTVs computer-generated "pseudo- 
film" images. A sequence of these "view- 
plane" images typically was generated 
to represent one or more "corridor" ap- 
proaches to a target area. The overhead 
source images in both applications were 
first orthorectified to remove any ter- 
rain elevation relief distortion and then 



mosaicked together to be in registration 
with the terrain elevation array. The 
terrain elevation array was then refor- 
matted into a mesh of triangles. It 
would be used in this form along with 
the registered orthophoto mosaic to gen- 
erate the requisite oblique "pseudo-film" 
or "viewplane" images for a desired 
gaming area. The rendering techniques 
used to create these oblique images 
were similar to those described in the 
preceding sections for nonreal-time, 
polygon-based, perspective scene gener- 
ation. Since this rendering was per- 
formed by a nonreal-time preprocessing 
step, there was no need to limit the 
number of polygons used to represent 
the terrain nor were there any restric- 
tions as to which anti-aliasing tech- 
niques could be used. Consequently, 
these oblique perspective images suf- 
fered no appreciable loss of quality or 
realism. 

In the late 1970s and early 1980s, 
another set of researchers, notably from 
General Electric and Evans & Suther- 
land, started developing direct tech- 
niques for real-time rendering of polyg- 
onal models of the terrain and urban 
features. These techniques were based 
upon the tools evolving from the com- 
puter graphics industry. In the begin- 
ning, a very limited number of polygo- 
nal surfaces were either simply 
assigned colors or were flat shaded ac- 
cording to an illumination model. Strict 
limits on the number of polygons that 
could be processed in any one output 
scene had to be maintained. In order to 
keep the polygon load to a minimum, 
these systems typically represented the 
terrain using a mesh of irregularly sized 
triangles, called a triangulated irregu- 
lar net or TIN. These TINs were often 
generated using the technique of Delau- 
nay triangulation [Lee and Schachter 
1980] or by a hierarchical subdivision 
technique [Bunker and Heartz 1976] 
starting from a regular grid of elevation 
values. The objective was to use larger 
triangles where the terrain was gener- 
ally smooth and finer triangles where 
the terrain undulated more rapidly. A 
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summary of many of these early flight 
simulator systems and techniques has 
been presented by Schachter and Ahuja 
[1980], Schachter [1981], and Yan 
[1985]. 

As both hardware and computer 
graphics technologies matured, the 

number of polygons processed in an out- 
put scene has increased to represent 
more faithfully the undulations of the 
terrain. Also, the flat shading technique 
was replaced by Gouraud and Phong 
interpolated shading techniques that 
smoothed over and disguised the faceted 
appearance of many of the 3D models. 
Eventually, the demand for more real- 
ism resulted in the application of syn- 
thetic and photo-derived textures. Some 
of the techniques used to generate the 
synthetic texture maps as well as the 
ways these texture maps were used to 
achieve various simulator effects have 
been summarized by Robinson and Zim- 
merman [1985]. For the photo-derived 
case, nongeographic-specific texture 
maps of typical terrain cover such as 
grassland, forests, marsh, and deserts 
were created from actual photographs of 
such areas. These generic texture maps 
were then "painted" onto the surfaces of 
the terrain polygons when needed to 
represent similar areas of the world. 
Later, actual photographs of geographic- 
specific regions were used to create real- 
time perspective fly-throughs of corre- 
sponding real-world regions. Usually, 
these photographs had to be orthorecti- 
fied, mosaicked together, and registered 
with the terrain elevation model before 
they could be used. Similarly, when 3D 
models of real-world urban features 
were to be included, most of the time 
specially acquired face-on photographs 
of each side of the object were used. 
Alternately, the texture patterns for the 
faces of the 3D models were extracted 
from oblique photographs and rectified 
into face-on views. Examples of this 
technology have been presented by Gen- 
eral Electric [Economy et al. 1988], 
Evans & Sutherland (Geer and Dixon 
1988], Star Technologies [Rich 1989] 
(now AAI Visual Systems), Thomsen 



CSF [Allain and Boidin 1986], and Boe- 
ing [Ross 1990]. 

Some of today's more advanced visual 
systems, such as Evans &; Sutherland's 
[1990] ESIG-4500 [Cosman et al. 1990; 
Abascal 1996], Lockheed-Martin's COM- 
PUSCENE VI [Abascal 1996], and 
Flight Safety International's VITAL 
VIII [Abascal 1996], generate images for 
cockpit display windows using mul- 
tiresolution, geographic-specific digital 
imagery. However, these real-time sys- 
tems still make compromises. Although 
many of these systems use supersam- 
pling or trilinear interpolation, aliasing 
artifacts, nonetheless, may occasionally 
be observed even with the use of mul- 
tiresolution data. This aliasing is pri- 
marily due to the difference in longitu- 
dinal (along the viewing direction) and 
transverse (across the viewing direc- 
tion) resolution requirements associated 
with highly oblique output perspective 
views of the orthophoto mosaic. This 
type of aliasing can be dramatically re- 
duced when techniques such as ellipti- 
cally weighted averaging are used for 
adaptive resampling; however, the com- 
putation burden associated with this 
technique currently is too high to main- 
tain real-time rates. Nevertheless, to 
the users familiar with generic photo- 
derived textures or monoresolution im- 
agery, the use of multiresolution data is 
a significant step forward. 

The data rates and memory require- 
ments of the multiresolution digital or- 
thophoto mosaic approach have been de- 
scribed in a paper by Hooks et al. 
[1990]. This paper assumed that, for a 
given output image eye point and field- 
of-view, all relevant pixel data from the 
orthophoto mosaic would be available at 
the appropriate resolution so that the 
resulting system was display resolution- 
limited. The authors presented formu- 
lae that characterized the data rates 
and memory requirements of such a sys- 
tem as functions of the number of differ- 
ent resolution levels used, the scale fac- 
tor between successive resolution levels, 
and the field-of-view corresponding to 
the output image. The calculations pre- 



ACM Computing Surveys, Vol. 29, No. 4, December 1997 



Texture Mapping 3D Models • 359 



sented in this article identified two very 
interesting facts: the optimum scahng 
factor between successive resolution 
levels is about 1.12 rather than 2; and 
simulations for sensors with narrow 
fields-of-view (i.e., a telephoto lens) and 
high slew rates place considerably more 
severe demands upon these real-time 
systems with respect to resolution, 
memory, and I/O bandwidth than do 
those for out-the-window displays. 

A variation of the "pseudo-film" tech- 
nique has been used by LTV to over- 
come this latter problem and to mitigate 
the omnidirectional filtering problem. 
In this case, the "pseudo-film" images 
were created from the orthophoto mo- 
saic to simulate a full 360-degree hori- 
zontal and 100-degree vertical field-of- 
view rather than one more typical of a 
planar central perspective image, 
namely, about 70 degrees in both di- 
mensions. During their generation, 
these fish-eye-view images were also 
prefiltered to have roughly the same 
resolution in both transverse and longi- 
tudinal directions. The real-time system 
then used the fish-eye-view "pseudo-im- 
ages" in conjunction with the polygonal- 
ized terrain elevation model to interpo- 
late the output images corresponding to 
the narrow field-of-view sensor's dis- 
play. This type of hybrid technique al- 
lowed full freedom of flight around the 
area of interest, produced high quality 
output images due to the off-line anti- 
aliasing, and maintained high data 
throughput rates associated with rapid 
slewing of narrow field-of-view sensors. 

As mentioned earlier, the worksta- 
tion-based nonreal-time rendering sys- 
tems and the entirely custom-built high 
performance real-time systems have 
been inching towards each other to the 
point where there no longer is a huge 
void of capability between the two. This 
is primarily due to SGFs series of high 
performance CPUs and gTaphics en- 
gines and to the Pixel-Planes/Pixel-Flow 
technology [Fuchs et al. 1985; Molnar et 
al. 1992] developed at the University of 
North Carolina and commercialized by 
Division Ltd. and IVEX Corporation. 



For instance, an SGI Onyx system (pref- 
erably with multiple CPUs) and a Real- 
ity Engine-2 or Infinite Reality graphics 
engine can provide real-time rendering 
of a high resolution mosaic that is 
stored in high-speed texture memory. 
This has been demonstrated by several 
third party companies, most notably by 
Cambridge Research Associates, Gemini 
Corp., Autometric Inc., and LMVS (on 
their TOPSCENE Model 4000). On the 
other side, Evans & Sutherland and 
Lockheed-Martin Co. (Orlando, FL) have 
been engaged in developing smaller and 
leaner versions of their custom-built 
Image Generators (IGs). For example, 
Lockheed-Martin's REAL3D, which is a 
derivative of their COMPUSCENE 
product, is now being used by the com- 
puter gaming industry. Also, Evans & 
Sutherland is now offering a worksta- 
tion-based system called iNTegrator. 
This system provides optional real-time 
rendering running on an Intel Pentium 
when accelerated by their custom Har- 
mony IGs. 

5, CONCLUSION 

The technology to generate arbitrary 
views of real-world areas from photo- 
graphic information has evolved from 
both the computer graphics and flight 
simulation industries. Two decades ago 
representations of real or fictitious 
scenes were generated using simple 
wire frames. Then illumination-based 
shading techniques were developed that 
over time have evolved to create in- 
creasingly more realistic effects. This 
was followed by the introduction of tex- 
ture mapping — a technique that adds 
realistic-looking synthetic or even pho- 
tographic texture patterns to faces of 
objects. Over a decade ago, the first true 
perspective transformations of images 
were published that completely texture 
mapped every 3D model in the scene 
with actual photographic information 
about the real-world model to which it 
corresponded. The popularity of this 
technology has grown dramatically over 
the last few years. Low-cost worksta- 
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tion-based systems exist today that can 
present moderate quality 2.5D trans- 
formed views of natural terrain environ- 
ments in near real-time (approximately 
one frame per second). This is achieved 
using modern, general-purpose proces- 
sors and efficient software, but limited 
texture information must be specially 
preprocessed to a vertical format, mosa- 
icked together, and then stored in mem- 
ory. Other low-cost, workstation-based 
systems are available that can create 
high quality 2.5D views for arbitrary 
mixes of natural terrain and cultural 
models. They can handle extremely 
large amounts of imagery with minimal 
preprocessing, but they are typically 
slower, since these data must be trans- 
ferred from disk storage. Furthermore, 
the algorithms must be general enough 
to handle urban models as well as the 
terrain and therefore are not quite as 
efficient. Commercial and military 
flight simulators as well as some high- 
end workstations can achieve real-time 
perspective transformation rates for 
mixed rural and urban areas, because 
they use special-purpose parallel hard- 
ware and make compromises with re- 
spect to algorithms and data loads. 

Rapid advances are currently being 
made in commercial computer technol- 
ogy in the areas of computational speed 
(e.g., superscalar chips) and texture 
memory capacity. Advances in disk and 
bus technology, however, have pro- 
gressed at a slightly slower pace. Never- 
theless, some advances have been 
achieved in the areas of formatting and 
storage of data for efficient processing. 
For example, multiresolution data and 
special tiling and paging strategies have 
become increasingly popular and neces- 
sary for enhanced rendering times. 
Other advances have been made in disk 
and disk controller technology. For ex- 
ample, disk arrays and parallel transfer 
disks (the so-called real-time disks) are 
available and have been used for sev- 
eral years on image processing systems. 
It is inevitable that all these limitations 
eventually will be solved and low-cost, 
well integrated, real-time, IPT worksta- 



tions based upon commercial technology 
will be available, perhaps even within 
the next few years. 
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^ ABSTRACT 



Texture nnapping has become a popular tool in the computer graphics industry in the last few years 
because it is an easy way to achieve a high degree of realism In computer-generated imagery with 
very little effort. Over the last decade, texture-mapping techniques have advanced to the point 
where it is possible to generate real-time perspective simulations of reahworld areas by texture 
mapping every object surface with texture from photographic images of these real-world areas. The 
techniques for generating such perspective transformations are variations on traditional texture 
mapping that in some circles have become known as the Image Perspective Transformation or IPT 
technology. This article first presents a background survey of traditional texture mapping. It then 
continues with a description of the texture-mapping variations that achieve these perspective 
transformations of photographic images of real-world scenes. The style of the presentation is that of 
a resource survey rather thatn an in-depth analysis. 
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The authors present a fairly in-depth survey of texture mapping. They review the traditional 
techniques used in the past ten years in geometric engines and visual simulators. They also cover 
the different variations that expand on the basic techniques In order to achieve realistic rendering. 
The paper is more of a resource survey, with a long list of references, than it is an in-depth analysis 
that will help readers to Implement a specific method. Its real value is in its wide coverage of 
techniques, with discussion of their specific fields of application, advantages, and drawbacks. In this 
sense, the paper can serve as a starting point for selection of the best approach to a particular 
problem. The authors emphasize image perspective transform (IPT), a technique that maps a large 
image texture (such as a photograph) onto geometric objects and respects the perspective 
transformation applied in the process. Several tables clarify the main points that differentiate the 
various techniques and provide the key transformation equations readers will encounter in 
traditional texture mapping techniques. Several pictures and figures illustrate the rendering 
techniques used, and numerous references to existing equipment help place the different efforts on 
a time and application spectrum. The authors also present both real-time and non-real-time IPT 
systems. This is a good reference for anyone who wants to learn about the techniques available 
today for mapping textures (either computer-generated or derived from the real world) onto 
geometric data. It also traces the history and evolution of texture mapping. Online Computin g 
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